mangos-mirror/server
0
0
Fork 0
mirror of https://github.com/mangosfour/server.git synced 2025-12-17 16:37:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

[10166] Romve now unused Stormlib, it also included a redundant copy of zlib.

Browse source 
Note that WoW 4.x support for libmpq still unclear, but in any case
this Stormlib version is outdated too and will not support it either,
even if we need to switch back.
This commit is contained in:
Lynx3d 2010-07-06 02:16:54 +02:00
parent 5e89098a61
commit 988ca2e101
189 changed files with 1 additions and 150507 deletions
Download patch file Download diff file Expand all files Collapse all files

318
contrib/vmap_extractor_v2/doc/The MoPaQ File Format.txt
View file

@ -1,318 +0,0 @@
THE MOPAQ ARCHIVE FORMAT
v0.9 (Thursday, June 30, 2005)
by Justin Olbrantz(Quantam)
Distribution and reproduction of this specification are allowed without limitation, as long as it is not altered. Quoting
in other works is freely allowed, as long as the source and author of the quote is stated.
TABLE OF CONTENTS
1. Introduction to the MoPaQ Format
2. The MoPaQ Format
2.1 General Archive Layout
2.2 Archive Header
2.3 Block Table
2.4 Hash Table
2.5 File Data
2.6 Listfile
2.7 Extended Attributes
2.8 Weak (Old) Digital Signature
2.9 Strong (New) Digital Signature
3. Algorithm Source Code
3.1 Encryption/Decryption
3.2 Hashing
3.3 Conversion of FILETIME and time_t
1. INTRODUCTION TO THE MOPAQ FORMAT
The MoPaQ (or MPQ) format is an archive file format designed by Mike O'Brien (hence the name Mike O'brien PaCK) at Blizzard
Entertainment. The format has been used in all Blizzard games since (and including) Diablo. It is heavily optimized to be
a read-only game archive format, and excels at this role.
The Blizzard MoPaQ-reading functions are contained in the Storm module, which my be either statically or dynamically linked.
The Blizzard MoPaQ-writing functions are contained in the MPQAPI module, which is always statically linked.
2. THE MOPAQ FORMAT
All numbers in the MoPaQ format are in little endian. Data types are listed either as int (integer, the number of bits specified),
byte (8 bits), and char (bytes which contain ASCII characters). All sizes and offsets are in bytes, unless specified otherwise.
Structure members are listed in the following general form:
offset from the beginning of the structure: data type(array size) member name : member description
2.1 GENERAL ARCHIVE LAYOUT
- Archive Header
- File Data
- File Data - Special Files
- Hash Table
- Block Table
- Strong Digital signature
This is the usual archive format, and is not absolutely essential. Some archives have been observed placing the hash table
and file table after the archive header, and before the file data.
2.2 ARCHIVE HEADER
00h: char(4) Magic : Indicates that the file is a MoPaQ archive. Must be ASCII "MPQ" 1Ah.
04h: int32 HeaderSize : Size of the archive header. Should be 32.
08h: int32 ArchiveSize : Size of the whole archive, including the header. Does not include the strong digital signature, if present.
This size is used, among other things, for determining the region to hash in computing the digital signature.
0Ch: int16 Unknown : Unknown
0Eh: int8 SectorSizeShift : Power of two exponent specifying the number of 512-byte disk sectors in each logical sector
in the archive. The size of each logical sector the archive is 512 * 2^SectorSizeShift. Bugs in the Storm library dictate
that this should always be 3 (4096 byte sectors).
10h: int32 HashTableOffset : Offset to the beginning of the hash table, relative to the beginning of the archive.
14h: int32 BlockTableOffset : Offset to the beginning of the block table, relative to the beginning of the archive.
18h: int32 HashTableEntries : Number of entries in the hash table. Must be a power of two, and must be less than 2^16.
1Ch: int32 BlockTableEntries : Number of entries in the block table.
The archive header is the first structure in the archive, at archive offset 0, but the archive does not need to be at offset
0 of the containing file. The offset of the archive in the file is referred to here as ArchiveOffset. If the archive is not
at the beginning of the file, it must begin at a disk sector boundary (512 bytes). Early versions of Storm require that the
archive be at the end of the containing file (ArchiveOffset + ArchiveSize = file size), but this is not required in newer
versions (due to the strong digital signature not being considered a part of the archive).
2.3 BLOCK TABLE
The block table contains entries for each region in the archive. Regions may be either files or empty space, which may be
overwritten by new files (typically this space is from deleted file data). The block table is encrypted, using the hash
of "(block table)" as the key. Each entry is structured as follows:
00h: int32 BlockOffset : Offset of the beginning of the block, relative to the beginning of the archive. Meaningless if the block size is 0.
04h: int32 BlockSize : Size of the block in the archive.
08h: int32 FileSize : Size of the file data stored in the block. Only valid if the block is a file, otherwise meaningless, and should be 0. If the file is compressed, this is the size of the uncompressed file data.
0Ch: int32 Flags : Bit mask of the flags for the block. The following values are conclusively identified:
80000000h: Block is a file, and follows the file data format; otherwise, block is free space, and may be overwritten. If the block is not a file, all other flags should be cleared.
01000000h: File is stored as a single unit, rather than split into sectors.
00020000h: The file's encryption key is adjusted by the block offset and file size (explained in detail in the File Data section). File must be encrypted.
00010000h: File is encrypted.
00000200h: File is compressed. Mutually exclusive to file imploded.
00000100h: File is imploded. Mutually exclusive to file compressed.
2.4 HASH TABLE
Instead of storing file names, for quick access MoPaQs use a fixed, power of two-size hash table of files in the archive. A file is uniquely identified by its file path, its language, and its platform. The home entry for a file in the hash table is computed as a hash of the file path. In the event of a collision (the home entry is occupied by another file), progressive overflow is used, and the file is placed in the next available hash table entry. Searches for a desired file in the hash table proceed from the home entry for the file until either the file is found, the entire hash table is searched, or an empty hash table entry (FileBlockIndex of FFFFFFFFh) is encountered. The hash table is encrypted using the hash of "(hash table)" as the key. Each entry is structured as follows:
00h: int32 FilePathHashA : The hash of the file path, using method A.
04h: int32 FilePathHashB : The hash of the file path, using method B.
08h: int16 Language : The language of the file. This is a Windows LANGID data type, and uses the same values. 0 indicates the default language (American English), or that the file is language-neutral.
0Ah: int8 Platform : The platform the file is used for. 0 indicates the default platform. No other values have been observed.
0Ch: int32 FileBlockIndex : If the hash table entry is valid, this is the index into the block table of the file. Otherwise, one of the following two values:
FFFFFFFFh: Hash table entry is empty, and has always been empty. Terminates searches for a given file.
FFFFFFFEh: Hash table entry is empty, but was valid at some point (in other words, the file was deleted). Does not terminate searches for a given file.
2.5 FILE DATA
00h: int32(SectorsInFile + 1) SectorOffsetTable : Offsets to the start of each sector's data, relative to the beginning of the file data. Not present if this information is calculatable (see details below).
immediately following SectorOffsetTable: SectorData : Data of each sector in the file, packed end to end (see details below).
Normally, file data is split up into sectors, for simple streaming. All sectors, save for the last, will contain as many bytes of file data as specified in the archive header's SectorSizeShift; the last sector may be smaller than this, depending on the size of the file data. This sector size is the size of the raw file data; if the file is compressed, the compressed sector will be smaller or the same size as the uncompressed sector size. Individual sectors in a compressed file may be stored uncompressed; this occurs if and only if the sector could not be compressed by the algorithm used (if the compressed sector size was greater than or equal to the size of the raw data), and is indicated by the sector's compressed size in SectorOffsetTable being equal to the uncompressed size of the sector (which may be calculated from the FileSize).
If the sector is compressed (but not imploded), a bit mask byte of the compression algorithm(s) used to compress the sector is appended to the beginning of the compressed sector data. This additional byte counts towards the total size of the sector; if the size of the sector (including this byte) exceeds or matches the uncompressed size of the sector data, the sector will be stored uncompressed, and this byte omitted. Multiple compression algorithms may be used on the same sector; in this case, successive compression occurs in the order the algorithms are listed below, and decompression occurs in the opposite order. For implimentations of all of these algorithms, see StormLib.
40h: IMA ADPCM mono
80h: IMA ADPCM stereo
01h: Huffman encoded
02h: Deflated (see ZLib)
08h: Imploded (see PKWare Data Compression Library)
10h: BZip2 compressed (see BZip2)
If the file is stored as a single unit (indicated in the file's Flags), there is effectively only a single sector, which
contains the entire file.
If the file is encrypted, each sector (after compression and appendage of the compression type byte, if applicable)
is encrypted with the file's key. The base key for a file is determined by a hash of the file name stripped of the
directory (i.e. the key for a file named "directory\file" would be computed as the hash of "file"). If this key is
adjusted, as indicated in the file's Flags, the final key is calculated as ((base key + BlockOffset - ArchiveOffset)
XOR FileSize) (StormLib - an open-source implementation of the MoPaQ reading and writing functions,
by Ladislav Zezula - incorrectly uses an AND in place of the XOR). Each sector is encrypted using the key + the
0-based index of the sector in the file. The SectorOffsetTable, if present, is encrypted using the key - 1.
The SectorOffsetTable is omitted when the sizes and offsets of all sectors in the file are calculatable from the FileSize.
This can happen in several circumstances. If the file is not compressed/imploded, then the size and offset of all sectors
is known, based on the archive's SectorSizeShift. If the file is stored as a single unit compressed/imploded, then the
SectorOffsetTable is omitted, as the single file "sector" corresponds to BlockSize and FileSize, as mentioned previously.
Note that the SectorOffsetTable will always be present if the file is compressed/imploded and the file is not stored as
a single unit, even if there is only a single sector in the file (the size of the file is less than or equal to the
archive's sector size).
2.6 LISTFILE
The listfile is a very simple extension to the MoPaQ format that contains the file paths of (most) files in the archive.
The languages and platforms of the files are not stored in the listfile. The listfile is contained in the file "(listfile)",
and is simply a non-Unix-style text file with one file path on each line, lines terminated with the bytes 0Dh 0Ah. The file
"(listfile)" may not be listed in the listfile.
2.7 EXTENDED ATTRIBUTES
The extended attributes are optional file attributes for files in the block table. These attributes were added at times after
the MoPaQ format was already finalized, and it is not necessary for every archive to have all (or any) of the extended attributes.
If an archive contains a given attribute, there will be an instance of that attribute for every block in the block table, although
the attribute will be meaningless if the block is not a file. The order of the attributes for blocks correspond to the order of the
blocks in the block table, and are of the same number. The attributes are stored in parallel arrays in the "(attributes)" file,
in the archive. The attributes corresponding to this file need not be valid (and logically cannot be). Unlike all the other
structures in the MoPaQ format, entries in the extended attributes are NOT guaranteed to be aligned. Also note that in some
archives, malicious zeroing of the attributes has been observed, perhaps with the intent of breaking archive viewers. This
file is structured as follows:
00h: int32 Version : Specifies the extended attributes format version. For now, must be 100.
04h: int32 AttributesPresent : Bit mask of the extended attributes present in the archive:
00000001h: File CRC32s.
00000002h: File timestamps.
00000004h: File MD5s.
08h: int32(BlockTableEntries) CRC32s : CRC32s of the (uncompressed) file data for each block in the archive. Omitted if the
archive does not have CRC32s. immediately after CRC32s: FILETIME(BlockTableEntries) Timestamps : Timestamps for each block
in the archive. The format is that of the Windows FILETIME structure. Omitted if the archive does not have timestamps.
immediately after Timestamps: MD5(BlockTableEntries) MD5s : MD5s of the (uncompressed) file data for each block in the archive.
Omitted if the archive does not have MD5s.
2.8 WEAK DIGITAL SIGNATURE
The weak digital signature is a digital signature using Microsoft CryptoAPI. It is an implimentation of the RSASSA-PKCS1-v1_5
digital signature protocol, using the MD5 hashing algorithm and a 512-bit (weak) RSA key (for more information about this
protocol, see the RSA Labs PKCS1 specification). The public key and exponent are stored in a resource in Storm. The signature
is stored uncompressed, unencrypted in the file "(signature)" in the archive. The archive is hashed from the beginning of the
archive (ArchiveOffset in the containing file) to the end of the archive (the length indicated by ArchiveSize); the signature
file is added to the archive before signing, and the space occupied by the file is considered to be all binary 0s during
signing/verification. This file is structured as follows:
00h: int32 Unknown : Must be 0.
04h: int32 Unknown : must be 0.
08h: int512 Signature : The digital signature. Like all other numbers in the MoPaQ format, this is stored in little-endian order.
2.9 STRONG DIGITAL SIGNATURE
The strong digital signature uses a simple proprietary implementation of RSA signing, using the SHA-1 hashing algorithm and
a 2048-bit (strong) RSA key. The default public key and exponent are stored in Storm, but other keys may be used as well.
The strong digital signature is stored immediately after the archive, in the containing file; the entire archive (ArchiveSize
bytes, starting at ArchiveOffset in the containing file) is hashed as a single block. The signature has the following format:
00h: char(4) Magic : Indicates the presence of a digital signature. Must be "NGIS" ("SIGN" backwards).
04h: int2048 Signature : The digital signature, stored in little-endian format.
When the Signature field is decrypted with the public key and exponent, and the result stored in little-endian order, it is structured as follows:
00h: byte Padding : Must be 0Bh.
01h: byte(235) Padding : Must be BBh.
ECh: byte(20) SHA-1 : SHA-1 hash of the archive, in standard SHA-1 format.
3. ALGORITHM SOURCE CODE
3.1 ENCRYPTION/DECRYPTION
I believe this was derived at some point from code in StormLib. Assumes the long type to be 32 bits, and the machine to be little endian order.
unsigned long dwCryptTable[0x500];
void InitializeCryptTable()
{
unsigned long seed = 0x00100001;
unsigned long index1 = 0;
unsigned long index2 = 0;
int i;
for (index1 = 0; index1 < 0x100; index1++)
{
for (index2 = index1, i = 0; i < 5; i++, index2 += 0x100)
{
unsigned long temp1, temp2;
seed = (seed * 125 + 3) % 0x2AAAAB;
temp1 = (seed & 0xFFFF) << 0x10;
seed = (seed * 125 + 3) % 0x2AAAAB;
temp2 = (seed & 0xFFFF);
dwCryptTable[index2] = (temp1 | temp2);
}
}
}
void EncryptData(void *lpbyBuffer, unsigned long dwLength, unsigned long dwKey)
{
unsigned long *lpdwBuffer = (unsigned long *)lpbyBuffer;
unsigned long seed = 0xEEEEEEEE;
unsigned long ch;
assert(lpbyBuffer);
dwLength /= sizeof(unsigned long);
while(dwLength-- > 0)
{
seed += dwCryptTable[0x400 + (dwKey & 0xFF)];
ch = *lpdwBuffer ^ (dwKey + seed);
dwKey = ((~dwKey << 0x15) + 0x11111111) | (dwKey >> 0x0B);
seed = *lpdwBuffer + seed + (seed << 5) + 3;
*lpdwBuffer++ = ch;
}
}
void DecryptData(void *lpbyBuffer, unsigned long dwLength, unsigned long dwKey)
{
unsigned long *lpdwBuffer = (unsigned long *)lpbyBuffer;
unsigned long seed = 0xEEEEEEEE;
unsigned long ch;
assert(lpbyBuffer);
dwLength /= sizeof(unsigned long);
while(dwLength-- > 0)
{
seed += dwCryptTable[0x400 + (dwKey & 0xFF)];
ch = *lpdwBuffer ^ (dwKey + seed);
dwKey = ((~dwKey << 0x15) + 0x11111111) | (dwKey >> 0x0B);
seed = ch + seed + (seed << 5) + 3;
*lpdwBuffer++ = ch;
}
}
3.2 HASHING
Based on code from StormLib.
// Different types of hashes to make with HashString
#define MPQ_HASH_TABLE_OFFSET 0
#define MPQ_HASH_NAME_A 1
#define MPQ_HASH_NAME_B 2
#define MPQ_HASH_FILE_KEY 3
unsigned long HashString(const char *lpszString, unsigned long dwHashType)
{
unsigned long seed1 = 0x7FED7FED;
unsigned long seed2 = 0xEEEEEEEE;
int ch;
while (*lpszString != 0)
{
ch = toupper(*lpszString++);
seed1 = dwCryptTable[(dwHashType * 0xFF) + ch] ^ (seed1 + seed2);
seed2 = ch + seed1 + seed2 + (seed2 << 5) + 3;
}
return seed1;
}
3.3 CONVERSION OF FILETIME AND time_t
#define EPOCH_OFFSET 116444736000000000ULL // Number of 100 ns units between 01/01/1601 and 01/01/1970
bool GetTimeFromFileTime(FILETIME &fileTime, time_t &time)
{
// The FILETIME represents a 64-bit integer: the number of 100 ns units since January 1, 1601
unsigned long long nTime = ((unsigned long long)fileTime.dwHighDateTime << 32) + fileTime.dwLowDateTime;
if (nTime < EPOCH_OFFSET)
return false;
nTime -= EPOCH_OFFSET; // Convert the time base from 01/01/1601 to 01/01/1970
nTime /= 10000000ULL; // Convert 100 ns to sec
time = (time_t)nTime;
// Test for overflow (FILETIME is 64 bits, time_t is 32 bits)
if ((nTime - (unsigned long long)time) > 0)
return false;
return true;
}
void GetFileTimeFromTime(time_t &time, FILETIME &fileTime)
{
unsigned long long nTime = (unsigned long long)time;
nTime *= 10000000ULL;
nTime += EPOCH_OFFSET;
fileTime.dwLowDateTime = (DWORD)nTime;
fileTime.dwHighDateTime = (DWORD)(nTime >> 32);
}

421
contrib/vmap_extractor_v2/doc/The_MoPaQ_File_Format.txt
View file

@ -1,421 +0,0 @@
THE MOPAQ ARCHIVE FORMAT
v1.0 (Friday, September 1, 2006)
by Justin Olbrantz(Quantam)
Distribution and reproduction of this specification are allowed without limitation, as long as it is not altered. Quotation in other works is freely allowed, as long as the source and author of the quote are stated.
TABLE OF CONTENTS
1. Introduction to the MoPaQ Format
2. The MoPaQ Format
2.1 General Archive Layout
2.2 Archive Header
2.3 Block Table
2.4 Extended Block Table
2.5 Hash Table
2.6 File Data
2.7 Listfile
2.8 Extended Attributes
2.9 Weak (Old) Digital Signature
2.10 Strong (New) Digital Signature
3. Algorithm Source Code
3.1 Encryption/Decryption
3.2 Hashing and File Key Computation
3.3 Finding Files
3.4 Deleting Files
3.5 Conversion of FILETIME and time_t
3.6 Forming a 64-bit Large Archive Offset from 32-bit and 16-bit Components
4. Revision History
1. INTRODUCTION TO THE MOPAQ FORMAT
The MoPaQ (or MPQ) format is an archive file format designed by Mike O'Brien (hence the name Mike O'brien PaCK) at Blizzard Entertainment. The format has been used in all Blizzard games since (and including) Diablo. It is heavily optimized to be a read-only game archive format, and excels at this role.
The Blizzard MoPaQ-reading functions are contained in the Storm module, which my be either statically or dynamically linked. The Blizzard MoPaQ-writing functions are contained in the MPQAPI module, which is always statically linked.
StormLib - mentioned several times in this specification - is an open-source MoPaQ reading and writing library written by Ladislav Zezula (no affiliation with Blizzard Entertainment). While it's a bit dated, and does not support all of the newer MoPaQ features, it contains source code to the more exotic compression methods used by MoPaQ, such as the PKWare implode algorithm, MoPaQ's huffman compression algorithm, and the IMA ADPCM compression used by MoPaQ.
2. THE MOPAQ FORMAT
All numbers in the MoPaQ format are in little endian byte order; signed numbers use the two's complement system. Data types are listed either as int (integer, the number of bits specified), byte (8 bits), or char (bytes which contain ASCII characters). All sizes and offsets are in bytes, unless specified otherwise. Structure members are listed in the following general form:
offset from the beginning of the structure: data type(array size) member name : member description
2.1 GENERAL ARCHIVE LAYOUT
- Archive Header
- File Data
- File Data - Special Files
- Hash Table
- Block Table
- Extended Block Table
- Strong Digital signature
This is the usual archive format, but it is not mandatory. Some archives have been observed placing the hash table and file table after the archive header, and before the file data.
2.2 ARCHIVE HEADER
00h: char(4) Magic : Indicates that the file is a MoPaQ archive. Must be ASCII "MPQ" 1Ah.
04h: int32 HeaderSize : Size of the archive header.
08h: int32 ArchiveSize : Size of the whole archive, including the header. Does not include the strong digital signature, if present. This size is used, among other things, for determining the region to hash in computing the digital signature. This field is deprecated in the Burning Crusade MoPaQ format, and the size of the archive is calculated as the size from the beginning of the archive to the end of the hash table, block table, or extended block table (whichever is largest).
0Ch: int16 FormatVersion : MoPaQ format version. MPQAPI will not open archives where this is negative. Known versions:
0000h: Original format. HeaderSize should be 20h, and large archives are not supported.
0001h: Burning Crusade format. Header size should be 2Ch, and large archives are supported.
0Eh: int8 SectorSizeShift : Power of two exponent specifying the number of 512-byte disk sectors in each logical sector in the archive. The size of each logical sector in the archive is 512 * 2^SectorSizeShift. Bugs in the Storm library dictate that this should always be 3 (4096 byte sectors).
10h: int32 HashTableOffset : Offset to the beginning of the hash table, relative to the beginning of the archive.
14h: int32 BlockTableOffset : Offset to the beginning of the block table, relative to the beginning of the archive.
18h: int32 HashTableEntries : Number of entries in the hash table. Must be a power of two, and must be less than 2^16 for the original MoPaQ format, or less than 2^20 for the Burning Crusade format.
1Ch: int32 BlockTableEntries : Number of entries in the block table.
Fields only present in the Burning Crusade format and later:
20h: int64 ExtendedBlockTableOffset : Offset to the beginning of the extended block table, relative to the beginning of the archive.
28h: int16 HashTableOffsetHigh : High 16 bits of the hash table offset for large archives.
2Ah: int16 BlockTableOffsetHigh : High 16 bits of the block table offset for large archives.
The archive header is the first structure in the archive, at archive offset 0; however, the archive does not need to be at offset 0 of the containing file. The offset of the archive in the file is referred to here as ArchiveOffset. If the archive is not at the beginning of the file, it must begin at a disk sector boundary (512 bytes). Early versions of Storm require that the archive be at the end of the containing file (ArchiveOffset + ArchiveSize = file size), but this is not required in newer versions (due to the strong digital signature not being considered a part of the archive).
2.3 BLOCK TABLE
The block table contains entries for each region in the archive. Regions may be either files, empty space, which may be overwritten by new files (typically this space is from deleted file data), or unused block table entries. Empty space entries should have BlockOffset and BlockSize nonzero, and FileSize and Flags zero; unused block table entries should have BlockSize, FileSize, and Flags zero. The block table is encrypted, using the hash of "(block table)" as the key. Each entry is structured as follows:
00h: int32 BlockOffset : Offset of the beginning of the block, relative to the beginning of the archive.
04h: int32 BlockSize : Size of the block in the archive.
08h: int32 FileSize : Size of the file data stored in the block. Only valid if the block is a file; otherwise meaningless, and should be 0. If the file is compressed, this is the size of the uncompressed file data.
0Ch: int32 Flags : Bit mask of the flags for the block. The following values are conclusively identified:
80000000h: Block is a file, and follows the file data format; otherwise, block is free space or unused. If the block is not a file, all other flags should be cleared, and FileSize should be 0.
01000000h: File is stored as a single unit, rather than split into sectors.
00020000h: The file's encryption key is adjusted by the block offset and file size (explained in detail in the File Data section). File must be encrypted.
00010000h: File is encrypted.
00000200h: File is compressed. File cannot be imploded.
00000100h: File is imploded. File cannot be compressed.
2.4 EXTENDED BLOCK TABLE
The extended block table was added to support archives larger than 4 gigabytes (2^32 bytes). The table contains the upper bits of the archive offsets for each block in the block table. It is simply an array of int16s, which become bits 32-47 of the archive offsets for each block, with bits 48-63 being zero. Individual blocks in the archive are still limited to 4 gigabytes in size. This table is only present in Burning Crusade format archives that exceed 4 gigabytes size.
As of the Burning Crusade Friends and Family beta, this table is not encrypted.
2.5 HASH TABLE
Instead of storing file names, for quick access MoPaQs use a fixed, power of two-size hash table of files in the archive. A file is uniquely identified by its file path, its language, and its platform. The home entry for a file in the hash table is computed as a hash of the file path. In the event of a collision (the home entry is occupied by another file), progressive overflow is used, and the file is placed in the next available hash table entry. Searches for a desired file in the hash table proceed from the home entry for the file until either the file is found, the entire hash table is searched, or an empty hash table entry (FileBlockIndex of FFFFFFFFh) is encountered. The hash table is encrypted using the hash of "(hash table)" as the key. Each entry is structured as follows:
00h: int32 FilePathHashA : The hash of the file path, using method A.
04h: int32 FilePathHashB : The hash of the file path, using method B.
08h: int16 Language : The language of the file. This is a Windows LANGID data type, and uses the same values. 0 indicates the default language (American English), or that the file is language-neutral.
0Ah: int8 Platform : The platform the file is used for. 0 indicates the default platform. No other values have been observed.
0Ch: int32 FileBlockIndex : If the hash table entry is valid, this is the index into the block table of the file. Otherwise, one of the following two values:
FFFFFFFFh: Hash table entry is empty, and has always been empty. Terminates searches for a given file.
FFFFFFFEh: Hash table entry is empty, but was valid at some point (in other words, the file was deleted). Does not terminate searches for a given file.
2.6 FILE DATA
The data for each file is composed of the following structure:
00h: int32(SectorsInFile + 1) SectorOffsetTable : Offsets to the start of each sector, relative to the beginning of the file data. The last entry contains the file size, making it possible to easily calculate the size of any given sector. This table is not present if this information can be calculated (see details below).
immediately following SectorOffsetTable: SECTOR Sectors(SectorsInFile) : Data of each sector in the file, packed end to end (see details below).
Normally, file data is split up into sectors, for simple streaming. All sectors, save for the last, will contain as many bytes of file data as specified in the archive header's SectorSizeShift; the last sector may contain less than this, depending on the size of the entire file's data. If the file is compressed or imploded, the sector will be smaller or the same size as the file data it contains. Individual sectors in a compressed or imploded file may be stored uncompressed; this occurs if and only if the file data the sector contains could not be compressed by the algorithm(s) used (if the compressed sector size was greater than or equal to the size of the file data), and is indicated by the sector's size in SectorOffsetTable being equal to the size of the file data in the sector (which may be calculated from the FileSize).
The format of each sector depends on the kind of sector it is. Uncompressed sectors are simply the the raw file data contained in the sector. Imploded sectors are the raw compressed data following compression with the implode algorithm (these sectors can only be in imploded files). Compressed sectors (only found in compressed - not imploded - files) are compressed with one or more compression algorithms, and have the following structure:
00h: byte CompressionMask : Mask of the compression types applied to this sector. If multiple compression types are used, they are applied in the order listed below, and decompression is performed in the opposite order. This byte counts towards the total sector size, meaning that the sector will be stored uncompressed if the data cannot be compressed by at least two bytes; as well, this byte is encrypted with the sector data, if applicable. The following compression types are defined (for implementations of these algorithms, see StormLib):
40h: IMA ADPCM mono
80h: IMA ADPCM stereo
01h: Huffman encoded
02h: Deflated (see ZLib)
08h: Imploded (see PKWare Data Compression Library)
10h: BZip2 compressed (see BZip2)
01h: byte(SectorSize - 1) SectorData : The compressed data for the sector.
If the file is stored as a single unit (indicated in the file's Flags), there is effectively only a single sector, which contains the entire file data.
If the file is encrypted, each sector (after compression/implosion, if applicable) is encrypted with the file's key. The base key for a file is determined by a hash of the file name stripped of the directory (i.e. the key for a file named "directory\file" would be computed as the hash of "file"). If this key is adjusted, as indicated in the file's Flags, the final key is calculated as ((base key + BlockOffset - ArchiveOffset) XOR FileSize) (StormLib incorrectly uses an AND in place of the XOR). Each sector is encrypted using the key + the 0-based index of the sector in the file. The SectorOffsetTable, if present, is encrypted using the key - 1.
The SectorOffsetTable is omitted when the sizes and offsets of all sectors in the file are calculatable from the FileSize. This can happen in several circumstances. If the file is not compressed/imploded, then the size and offset of all sectors is known, based on the archive's SectorSizeShift. If the file is stored as a single unit compressed/imploded, then the SectorOffsetTable is omitted, as the single file "sector" corresponds to BlockSize and FileSize, as mentioned previously. However, the SectorOffsetTable will be present if the file is compressed/imploded and the file is not stored as a single unit, even if there is only a single sector in the file (the size of the file is less than or equal to the archive's sector size).
2.7 LISTFILE
The listfile is a very simple extension to the MoPaQ format that contains the file paths of (most) files in the archive. The languages and platforms of the files are not stored in the listfile. The listfile is contained in the file "(listfile)" (default language and platform), and is simply a text file with file paths separated by ';', 0Dh, 0Ah, or some combination of these. The file "(listfile)" may not be listed in the listfile.
2.8 EXTENDED ATTRIBUTES
The extended attributes are optional file attributes for files in the block table. These attributes were added at times after the MoPaQ format was already finalized, and it is not necessary for every archive to have all (or any) of the extended attributes. If an archive contains a given attribute, there will be an instance of that attribute for every block in the block table, although the attribute will be meaningless if the block is not a file. The order of the attributes for blocks correspond to the order of the blocks in the block table, and are of the same number. The attributes are stored in parallel arrays in the "(attributes)" file (default language and platform), in the archive. The attributes corresponding to this file need not be valid (and logically cannot be). Unlike all the other structures in the MoPaQ format, entries in the extended attributes are NOT guaranteed to be aligned. Also note that in some archives, malicious zeroing of the attributes has been observed, perhaps with the intent of breaking archive viewers. This file is structured as follows:
00h: int32 Version : Specifies the extended attributes format version. For now, must be 100.
04h: int32 AttributesPresent : Bit mask of the extended attributes present in the archive:
00000001h: File CRC32s.
00000002h: File timestamps.
00000004h: File MD5s.
08h: int32(BlockTableEntries) CRC32s : CRC32s of the (uncompressed) file data for each block in the archive. Omitted if the archive does not have CRC32s.
immediately after CRC32s: FILETIME(BlockTableEntries) Timestamps : Timestamps for each block in the archive. The format is that of the Windows FILETIME structure. Omitted if the archive does not have timestamps.
immediately after Timestamps: MD5(BlockTableEntries) MD5s : MD5s of the (uncompressed) file data for each block in the archive. Omitted if the archive does not have MD5s.
2.9 WEAK DIGITAL SIGNATURE
The weak digital signature is a digital signature using Microsoft CryptoAPI. It is an implimentation of the RSASSA-PKCS1-v1_5 digital signature protocol, using the MD5 hashing algorithm and a 512-bit (weak) RSA key (for more information about this protocol, see the RSA Labs PKCS1 specification). The public key and exponent are stored in a resource in Storm, the private key is stored in a separate file, whose filename is passed to MPQAPI (the private key is not stored in MPQAPI). The signature is stored uncompressed, unencrypted in the file "(signature)" (default language and platform) in the archive. The archive is hashed from the beginning of the archive (ArchiveOffset in the containing file) to the end of the archive (the length indicated by ArchiveSize, or calculated in the Burning Crusade MoPaQ format); the signature file is added to the archive before signing, and the space occupied by the file is considered to be all binary 0s during signing/verification. This file is structured as follows:
00h: int32 Unknown : Must be 0.
04h: int32 Unknown : Must be 0.
08h: int512 Signature : The digital signature. Like all other numbers in the MoPaQ format, this is stored in little-endian order. The structure of this, when decrypted, follows the RSASSA-PKCS1-v1_5 specification; this format is rather icky to work with (I wrote a program to verify this signature using nothing but an MD5 function and huge integer functions; it wasn't pleasant), and best left to an encryption library such as Cryto++.
2.10 STRONG DIGITAL SIGNATURE
The strong digital signature uses a simple proprietary implementation of RSA signing, using the SHA-1 hashing algorithm and a 2048-bit (strong) RSA key. The default public key and exponent are stored in Storm, but other keys may be used as well. The strong digital signature is stored immediately after the archive, in the containing file; the entire archive (ArchiveSize bytes, starting at ArchiveOffset in the containing file) is hashed as a single block. The signature has the following format:
00h: char(4) Magic : Indicates the presence of a digital signature. Must be "NGIS" ("SIGN" backwards).
04h: int2048 Signature : The digital signature, stored in little-endian format.
When the Signature field is decrypted with the public key and exponent, and the resulting large integer is stored in little-endian order, it is structured as follows:
00h: byte Padding : Must be 0Bh.
01h: byte(235) Padding : Must be BBh.
ECh: byte(20) SHA-1 : SHA-1 hash of the archive, in standard SHA-1 byte order.
3. ALGORITHM SOURCE CODE
All of the sample code here assumes little endian machine byte order, that the short type is 16 bits, that the long type is 32 bits, and that the long long type is 64 bits. Adjustments must be made if these assumptions are not correct on a given platform. All code not credited otherwise was written by myself in the writing of this specification.
3.1 ENCRYPTION/DECRYPTION
Based on code from StormLib.
unsigned long dwCryptTable[0x500];
// The encryption and hashing functions use a number table in their procedures. This table must be initialized before the functions are called the first time.
void InitializeCryptTable()
{
unsigned long seed = 0x00100001;
unsigned long index1 = 0;
unsigned long index2 = 0;
int i;
for (index1 = 0; index1 < 0x100; index1++)
{
for (index2 = index1, i = 0; i < 5; i++, index2 += 0x100)
{
unsigned long temp1, temp2;
seed = (seed * 125 + 3) % 0x2AAAAB;
temp1 = (seed & 0xFFFF) << 0x10;
seed = (seed * 125 + 3) % 0x2AAAAB;
temp2 = (seed & 0xFFFF);
dwCryptTable[index2] = (temp1 | temp2);
}
}
}
void EncryptData(void *lpbyBuffer, unsigned long dwLength, unsigned long dwKey)
{
assert(lpbyBuffer);
unsigned long *lpdwBuffer = (unsigned long *)lpbyBuffer;
unsigned long seed = 0xEEEEEEEE;
unsigned long ch;
dwLength /= sizeof(unsigned long);
while(dwLength-- > 0)
{
seed += dwCryptTable[0x400 + (dwKey & 0xFF)];
ch = *lpdwBuffer ^ (dwKey + seed);
dwKey = ((~dwKey << 0x15) + 0x11111111) | (dwKey >> 0x0B);
seed = *lpdwBuffer + seed + (seed << 5) + 3;
*lpdwBuffer++ = ch;
}
}
void DecryptData(void *lpbyBuffer, unsigned long dwLength, unsigned long dwKey)
{
assert(lpbyBuffer);
unsigned long *lpdwBuffer = (unsigned long *)lpbyBuffer;
unsigned long seed = 0xEEEEEEEEL;
unsigned long ch;
dwLength /= sizeof(unsigned long);
while(dwLength-- > 0)
{
seed += dwCryptTable[0x400 + (dwKey & 0xFF)];
ch = *lpdwBuffer ^ (dwKey + seed);
dwKey = ((~dwKey << 0x15) + 0x11111111L) | (dwKey >> 0x0B);
seed = ch + seed + (seed << 5) + 3;
*lpdwBuffer++ = ch;
}
}
3.2 HASHING AND FILE KEY COMPUTATION
These functions may have been derived from StormLib code at some point in the very distant past. It was so long ago that I don't remember for certain.
// Different types of hashes to make with HashString
#define MPQ_HASH_TABLE_OFFSET 0
#define MPQ_HASH_NAME_A 1
#define MPQ_HASH_NAME_B 2
#define MPQ_HASH_FILE_KEY 3
// Based on code from StormLib.
unsigned long HashString(const char *lpszString, unsigned long dwHashType)
{
assert(lpszString);
assert(dwHashType <= MPQ_HASH_FILE_KEY);
unsigned long seed1 = 0x7FED7FEDL;
unsigned long seed2 = 0xEEEEEEEEL;
int ch;
while (*lpszString != 0)
{
ch = toupper(*lpszString++);
seed1 = dwCryptTable[(dwHashType * 0x100) + ch] ^ (seed1 + seed2);
seed2 = ch + seed1 + seed2 + (seed2 << 5) + 3;
}
return seed1;
}
#define BLOCK_OFFSET_ADJUSTED_KEY 0x00020000L
unsigned long ComputeFileKey(const char *lpszFilePath, const BlockTableEntry &blockEntry, unsigned long nArchiveOffset)
{
assert(lpszFilePath);
// Find the file name part of the path
const char *lpszFileName = strrchr(lpszFilePath, '\\');
if (lpszFileName)
lpszFileName++; // Skip the \
else
lpszFileName = lpszFilePath;
// Hash the name to get the base key
unsigned long nFileKey = HashString(lpszFileName, MPQ_HASH_FILE_KEY);
// Offset-adjust the key if necessary
if (blockEntry.Flags & BLOCK_OFFSET_ADJUSTED_KEY)
nFileKey = (nFileKey + blockEntry.BlockOffset) ^ blockEntry.FileSize;
return nFileKey;
}
3.3 FINDING FILES
#define MPQ_HASH_ENTRY_EMPTY 0xFFFFFFFFL
#define MPQ_HASH_ENTRY_DELETED 0xFFFFFFFEL
bool FindFileInHashTable(const HashTableEntry *lpHashTable, unsigned long nHashTableSize, const char *lpszFilePath, unsigned short nLang, unsigned char nPlatform, unsigned long &iFileHashEntry)
{
assert(lpHashTable);
assert(nHashTableSize);
assert(lpszFilePath);
// Find the home entry in the hash table for the file
unsigned long iInitEntry = HashString(lpszFilePath, MPQ_HASH_TABLE_OFFSET) & (nHashTableSize - 1);
// Is there anything there at all?
if (lpHashTable[iInitEntry].FileBlockIndex == MPQ_HASH_ENTRY_EMPTY)
return false;
// Compute the hashes to compare the hash table entry against
unsigned long nNameHashA = HashString(lpszFilePath, MPQ_HASH_NAME_A),
nNameHashB = HashString(lpszFilePath, MPQ_HASH_NAME_B),
iCurEntry = iInitEntry;
// Check each entry in the hash table till a termination point is reached
do
{
if (lpHashTable[iCurEntry].FileBlockIndex != MPQ_HASH_ENTRY_DELETED)
{
if (lpHashTable[iCurEntry].FilePathHashA == nNameHashA
&& lpHashTable[iCurEntry].FilePathHashB == nNameHashB
&& lpHashTable[iCurEntry].Language == nLang
&& lpHashTable[iCurEntry].Platform == nPlatform)
{
iFileHashEntry = iCurEntry;
return true;
}
}
iCurEntry = (iCurEntry + 1) & (nHashTableSize - 1);
} while (iCurEntry != iInitEntry && lpHashTable[iCurEntry].FileBlockIndex != MPQ_HASH_ENTRY_EMPTY);
return false;
}
3.4 DELETING FILES
bool DeleteFile(HashTableEntry *lpHashTable, unsigned long nHashTableSize, BlockTableEntry *lpBlockTable, const char *lpszFilePath, unsigned short nLang, unsigned char nPlatform)
{
assert(lpHashTable);
assert(nHashTableSize);
assert(lpBlockTable);
// Find the file in the hash table
unsigned long iFileHashEntry;
if (!FindFileInHashTable(lpHashTable, nHashTableSize, lpszFilePath, nLang, nPlatform, iFileHashEntry))
return false;
// Get the block table index before we nuke the hash table entry
unsigned long iFileBlockEntry = lpHashTable[iFileHashEntry].FileBlockIndex;
// Delete the file's entry in the hash table
memset(&lpHashTable[iFileHashEntry], 0xFF, sizeof(HashTableEntry));
// If the next entry is empty, mark this one as empty; otherwise, mark this as deleted.
if (lpHashTable[(iFileHashEntry + 1) & (nHashTableSize - 1)].FileBlockIndex == MPQ_HASH_ENTRY_EMPTY)
lpHashTable[iFileHashEntry].FileBlockIndex = MPQ_HASH_ENTRY_EMPTY;
else
lpHashTable[iFileHashEntry].FileBlockIndex = MPQ_HASH_ENTRY_DELETED;
// If the block occupies space, mark the block as free space; otherwise, clear the block table entry.
if (lpBlockTable[iFileBlockEntry].BlockSize > 0)
{
lpBlockTable[iFileBlockEntry].FileSize = 0;
lpBlockTable[iFileBlockEntry].Flags = 0;
}
else
memset(&lpBlockTable[iFileBlockEntry], 0, sizeof(BlockTableEntry);
return true;
}
3.5 CONVERSION OF FILETIME AND time_t
This code assumes that the base ("zero") date for time_t is 01/01/1970. This is true on Windows, Unix System V systems, and Mac OS X. It is unknown whether this is true on all other platforms. You'll need to research this yourself, if you plan on porting it somewhere else.
#define EPOCH_OFFSET 116444736000000000ULL // Number of 100 ns units between 01/01/1601 and 01/01/1970
bool GetTimeFromFileTime(const FILETIME &fileTime, time_t &time)
{
// The FILETIME represents a 64-bit integer: the number of 100 ns units since January 1, 1601
unsigned long long nTime = ((unsigned long long)fileTime.dwHighDateTime << 32) + fileTime.dwLowDateTime;
if (nTime < EPOCH_OFFSET)
return false;
nTime -= EPOCH_OFFSET; // Convert the time base from 01/01/1601 to 01/01/1970
nTime /= 10000000ULL; // Convert 100 ns to sec
time = (time_t)nTime;
// Test for overflow (FILETIME is 64 bits, time_t is 32 bits)
if ((nTime - (unsigned long long)time) > 0)
return false;
return true;
}
void GetFileTimeFromTime(const time_t &time, FILETIME &fileTime)
{
unsigned long long nTime = (unsigned long long)time;
nTime *= 10000000ULL;
nTime += EPOCH_OFFSET;
fileTime.dwLowDateTime = (DWORD)nTime;
fileTime.dwHighDateTime = (DWORD)(nTime >> 32);
}
3.6 FORMING A 64-BIT LARGE ARCHIVE OFFSET FROM 32-BIT AND 16-BIT COMPONENTS
unsigned long long MakeLargeArchiveOffset(unsigned long nOffsetLow, unsigned short nOffsetHigh)
{
return ((unsigned long long)nOffsetHigh << 32) + (unsigned long long)nOffsetLow;
}
4. REVISION HISTORY
1.0
- Updated to include most of the changes found in the Burning Crusade Friends and Family beta
0.91.
- Updated several structure member descriptions
- Listed the full set of characters that can separate list file entries
- Noted that (attributes), (listfile), and (signature) use the default language and platform codes
- Redid part of the file data specs to clarify the format of sectors
- Enhanced descriptions of the different kinds of block table entries
- Added ComputeFileKey, FindFileInHashTable, and DeleteFile source

117
contrib/vmap_extractor_v2/stormdll/StormDll.cpp
View file

@ -1,117 +0,0 @@
/*****************************************************************************/
/* Storm.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* This is just a dummy module for building import library for Storm.dll */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 11.04.03 1.00 Lad The first version of Storm.cpp */
/*****************************************************************************/
#include <windows.h>
#define BUILDING_STORM_CPP
#define STORM_ALTERNATE_NAMES
#include "StormDll.h"
BOOL WINAPI SFILE(OpenArchive)(LPCSTR lpFileName, DWORD dwPriority, DWORD dwFlags, HANDLE *hMPQ)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(CloseArchive)(HANDLE hMPQ)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(GetArchiveName)(HANDLE hMPQ, LPCSTR lpBuffer, DWORD dwBufferLength)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(OpenFile)(LPCSTR lpFileName, HANDLE *hFile)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(OpenFileEx)(HANDLE hMPQ, LPCSTR lpFileName, DWORD dwSearchScope, HANDLE *hFile)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(CloseFile)(HANDLE hFile)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
DWORD WINAPI SFILE(GetFileSize)(HANDLE hFile, LPDWORD lpFileSizeHigh)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(GetFileArchive)(HANDLE hFile, HANDLE *hMPQ)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(GetFileName)(HANDLE hFile, LPCSTR lpBuffer, DWORD dwBufferLength)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
DWORD WINAPI SFILE(SetFilePointer)(HANDLE hFile, long lDistanceToMove, PLONG lplDistanceToMoveHigh, DWORD dwMoveMethod)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(ReadFile)(HANDLE hFile,LPVOID lpBuffer,DWORD nNumberOfBytesToRead,LPDWORD lpNumberOfBytesRead,LPOVERLAPPED lpOverlapped)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
LCID WINAPI SFILE(SetLocale)(LCID nNewLocale)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(GetBasePath)(LPCSTR lpBuffer, DWORD dwBufferLength)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(SetBasePath)(LPCSTR lpNewBasePath)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SFILE(Destroy)()
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SCOMP(Compress)(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int uCmp, int uCmpType, int nCmpLevel)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}
BOOL WINAPI SCOMP(Decompress)(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
SetLastError(ERROR_CALL_NOT_IMPLEMENTED);
return FALSE;
}

25
contrib/vmap_extractor_v2/stormdll/StormDll.def
View file

@ -1,25 +0,0 @@
; Storm definition file with alternate Storm.dll names
LIBRARY "Storm"
EXPORTS
StormCloseArchive @252 ; 0x0FC
StormCloseFile @253 ; 0x0FD
StormDestroy @262 ; 0x106
StormGetFileArchive @264 ; 0x108
StormGetFileSize @265 ; 0x109
StormOpenArchive @266 ; 0x10A
StormOpenFile @267 ; 0x10B
StormOpenFileEx @268 ; 0x10C
StormReadFile @269 ; 0x10D
StormSetBasePath @270 ; 0x10E
StormSetFilePointer @271 ; 0x10F
StormSetLocale @272 ; 0x110
StormGetBasePath @273 ; 0x111
StormGetArchiveName @275 ; 0x113
StormGetFileName @276 ; 0x114
; StormSetLastError @465 ; 0x
StormCompress @551 ; 0x227
StormDecompress @552 ; 0x228

67
contrib/vmap_extractor_v2/stormdll/StormDll.h
View file

@ -1,67 +0,0 @@
/*****************************************************************************/
/* Storm.h Copyright Justin Olbrantz(Quantam) 2000 */
/*---------------------------------------------------------------------------*/
/* Storm Interface Library v1.0 for Windows */
/* */
/* Author : Justin Olbrantz(Quantam) */
/* E-mail : omega@dragonfire.net */
/* WWW : www.campaigncreations.com/starcraft/mpq2k/inside_mopaq/ */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.00 1.00 Qua The first version of Storm.h */
/* 11.04.03 1.00 Lad Added some functions */
/*****************************************************************************/
// We need the Windows data types for the Storm prototypes
#include <windows.h>
#ifndef __STORM_H__
#define __STORM_H__
// Somethimes is necessary to change the function names so they
// will not conflict with other MPQ tools.
#ifdef STORM_ALTERNATE_NAMES
#define SFILE(Name) Storm##Name
#define SCOMP(Name) Storm##Name
#else
#define SFILE(Name) SFile##Name
#define SCOMP(Name) SComp##Name
#endif
// Just in case anyone is still using C out there
#ifdef __cplusplus
extern "C" {
#endif
// Storm file function prototypes
BOOL WINAPI SFILE(OpenArchive)(LPCSTR lpFileName, DWORD dwPriority, DWORD dwFlags, HANDLE *hMPQ);
BOOL WINAPI SFILE(CloseArchive)(HANDLE hMPQ);
BOOL WINAPI SFILE(GetArchiveName)(HANDLE hMPQ, LPCSTR lpBuffer, DWORD dwBufferLength);
BOOL WINAPI SFILE(OpenFile)(LPCSTR lpFileName, HANDLE *hFile);
BOOL WINAPI SFILE(OpenFileEx)(HANDLE hMPQ, LPCSTR lpFileName, DWORD dwSearchScope, HANDLE *hFile);
BOOL WINAPI SFILE(CloseFile)(HANDLE hFile);
DWORD WINAPI SFILE(GetFileSize)(HANDLE hFile, LPDWORD lpFileSizeHigh);
BOOL WINAPI SFILE(GetFileArchive)(HANDLE hFile, HANDLE *hMPQ);
BOOL WINAPI SFILE(GetFileName)(HANDLE hFile, LPCSTR lpBuffer, DWORD dwBufferLength);
DWORD WINAPI SFILE(SetFilePointer)(HANDLE hFile, long lDistanceToMove, PLONG lplDistanceToMoveHigh, DWORD dwMoveMethod);
BOOL WINAPI SFILE(ReadFile)(HANDLE hFile,LPVOID lpBuffer,DWORD nNumberOfBytesToRead,LPDWORD lpNumberOfBytesRead,LPOVERLAPPED lpOverlapped);
LCID WINAPI SFILE(SetLocale)(LCID nNewLocale);
BOOL WINAPI SFILE(GetBasePath)(LPCSTR lpBuffer, DWORD dwBufferLength);
BOOL WINAPI SFILE(SetBasePath)(LPCSTR lpNewBasePath);
// Storm (de)compression functions
BOOL WINAPI SCOMP(Compress) (char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int uCmp, int uCmpType, int nCmpLevel);
BOOL WINAPI SCOMP(Decompress)(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength);
#if defined(_MSC_VER) && !defined(BUILDING_STORM_CPP)
#pragma comment(lib, "Storm.lib") // Force linking Storm.lib and thus Storm.dll
#endif
#ifdef __cplusplus
}
#endif
#endif // __STORM_H__

697
contrib/vmap_extractor_v2/stormlib/GfxDecode.cpp
View file

@ -1,697 +0,0 @@
/***********************************************************************
*
* Description: GfxDecode -- functions for reading Diablo's GFX files
* Author: Marko Friedemann <marko.friedemann@bmx-chemnitz.de>
* Created at: Son Jan 27 15:20:43 CET 2002
* Computer: hangloose.flachland-chemnitz.de
* System: Linux 2.4.16 on i686
*
* Copyright (c) 2002 BMX-Chemnitz.DE All rights reserved.
*
* ---------------------------------------------------------------------
* included are functions for getting:
* - the framecount of .CEL-files -> celGetFrameCount()
* - single frames of .CEL-files -> celGetFrameData()
* - the framecount of .CL2-files -> cl2GetFrameCount()
* - single directions of .CL2-files (all frames) -> cl2GetDirData()
* - single .PCX-files (256 color; v2, v5) -> pcxGetData()
***********************************************************************/
#include <vector>
#include <cmath>
#include <iostream>
#include "StormLib.h"
#define TRANS_COL 256
using std::cerr;
using std::vector;
/****** RAMP stuff *****************************************************
* for a more detailed description/explanation see below
***********************************************************************/
// two variations: one/two ramp(s)
static const uint16_t c_2RampSize = 544; // the frame size
static const uint16_t c_1RampSize = 800; // the frame size
// ramps (both variations) can be either left or right
static const uint16_t c_RampOffsetLeft[17] = {
0, // __
8, // + 8 note that this __--
24, // + 16 "drawing" is __--
48, // + 24 upside down! __-- this area
80, // + 32 __-- is always
120, // + 40 __-- colored
168, // + 48 __--
224, // + 56 __-- lower ramp ends here (+30 == 254)
288, // + 64 --__ upper ramp might be missing
348, // + 60 | --__
400, // + 52 | --__ this area
444, // + 44 | --__ is always
480, // + 36 | --__ colored
508, // + 28 | either trans- --__
528, // + 20 | parent or colored --__
540, // + 12 | --__ +2 Pixels = 544
542 // + 2 | this last one doesn't exist, it's those^ 2 pixels
};
static const uint16_t c_RampOffsetRight[17] = {
2, // __ before this, there are 2 Pixels
14, // + 12 --__ 4^2 - 2
34, // + 20 --__ 6^2 - 2
62, // + 28 this area --__ 8^2 - 2
98, // + 36 is always --__ 10^2 - 2 pattern anyone? ;)
142, // + 44 colored --__
194, // + 52 --__
254, // + 60 lower ramp ends here --__ (-30 == 224)
318, // + 64 upper ramp might be missing __--
374, // + 56 __-- |
422, // + 48 this area __-- | note that this
462, // + 40 is always __-- | "drawing"
494, // + 32 colored __-- eiter trans- | is upside down!
518, // + 24 __-- parent or colored |
534, // + 16 __-- |
542, // + 8 __-- +2 Startpixels = 544 |
542 // + 0 this last one doesn't exist, it | would be EOF
};
/****** FrameBuffer class **********************************************
* holds buffers and size information of the actual target image
* purpose: buffer management and avoidance of ugly globals
**********************************************************************/
class FrameBuf
{
protected:
vector <uint8_t *> vecData;
uint8_t *pCurrRow;
uint8_t *pPalette;
uint16_t uRows;
uint16_t *upYSize;
uint16_t *upMaxX;
public:
uint16_t uCols;
uint16_t uXSize;
uint16_t uMaxBlock;
uint16_t uFrameNum;
bool bHasBlocks;
bool bHasRamps;
FrameBuf(
uint8_t *pPal=NULL, uint16_t frame=0, uint16_t xsize=0,
uint16_t *pysize=NULL, uint16_t *pmaxx=NULL, uint16_t maxblock=0)
{
pCurrRow = new uint8_t[4*xsize];
pPalette = pPal;
uCols = 0;
uRows = 0;
uXSize = xsize;
uFrameNum = frame;
uMaxBlock = maxblock;
upYSize = pysize;
upMaxX = pmaxx;
bHasBlocks= false;
bHasRamps = false;
}
~FrameBuf()
{
delete[] pCurrRow;
for (vector <uint8_t *>::iterator vi=vecData.begin(); vi!=vecData.end(); vi++)
delete[] *vi;
vecData.clear();
}
void addLine()
{
++uRows;
uCols = 0;
vecData.push_back(pCurrRow);
pCurrRow = new uint8_t[4*uXSize];
}
void addPixel(uint16_t uColorNum)
{
if (uColorNum > TRANS_COL) {
cerr << "\n*** there seemed to be an error, illegal color index " << uColorNum;
cerr << "\n +++ at (" << uCols << "," << uRows << "), see for yourself *** \n\n";
uColorNum = TRANS_COL; // sane setting to avoid segfaults
}
memcpy(pCurrRow + 4*uCols, pPalette + 4*uColorNum, 4*sizeof(uint8_t));
if (++uCols == uXSize)
addLine();
else if ((uColorNum != TRANS_COL) && (upMaxX != NULL) && (uCols > *upMaxX))
*upMaxX = uCols;
}
// used to return the actual image data
uint8_t *getData()
{
uint16_t i;
vector <uint8_t *>::reverse_iterator vri;
// allocate a buffer to hold the actual image size
uint8_t *tmp = new uint8_t[4*uXSize*uRows];
// the lines are upside down inside the vector, use reverse iterator
for (i=0, vri=vecData.rbegin(); vri!=vecData.rend(); vri++, i++)
memcpy(tmp+4*uXSize*i, *vri, 4*uXSize*sizeof(uint8_t));
*upYSize = uRows; // set height
if (uCols > 0) {
cerr << "\n*** there seemed to be an error (last line does not match boundary, " << uCols << " pixels left)";
cerr << "\n +++ this is often caused by specifying an invalid width, see for yourself *** \n\n";
}
return tmp;
}
};
uint16_t WINAPI celGetFrameCount(uint8_t *pFileBuf)
{
uint32_t tmp;
memcpy(&tmp, pFileBuf, sizeof(uint32_t));
return (uint16_t)tmp;
}
/***** Block Decoder ***************************************************
* one of three possible decoding techniques necessary for .cel
* possible block contents are either colored (in that case the
* appropriate number of pixels are read) or transparent pixels
* there are neither ramps nor plain pixels allowed here
***********************************************************************/
uint8_t *celDecodeBlocks(uint8_t *pFileBuf, FrameBuf *pFrame, uint32_t *framestart)
{
uint32_t uFilePos=framestart[pFrame->uFrameNum];
uint8_t cRead=0, i=0;
if (!pFrame->bHasBlocks) // sanity check
return NULL;
while (uFilePos < framestart[pFrame->uFrameNum+1]) {
cRead = pFileBuf[uFilePos++];
if ((uFilePos == framestart[pFrame->uFrameNum]+1))
// TODO: what is this 0x0A 0x00 stuff all about?
if ((cRead == 0x0A) && (pFileBuf[uFilePos] == 0x00)) {
uFilePos += 9;
cRead = pFileBuf[uFilePos++];
}
if (cRead > 0x7F)
// transparent block (complement, 256-val)
for (i=0; i<256-cRead; i++)
pFrame->addPixel(TRANS_COL);
else if (cRead < pFrame->uMaxBlock + 1)
// pixel block (block size pixels to be read!)
for (i=0; i<cRead; i++)
pFrame->addPixel(pFileBuf[uFilePos++]);
else
cerr << "\n*** block mode: illegal block (> max_size) ***\n\n";
}
return pFrame->getData();
}
/***** Ramp Decoder ****************************************************
* the second of three possible decoding techniques necessary for .cel
* each block save the first/last is enclosed by two 0x00 pairs
* those blocks affect _TWO_ rows with one being 2 colored pixels shorter
* the first/last "block" affects only one row
***********************************************************************/
uint8_t *celDecodeRamps(uint8_t *pFileBuf, FrameBuf *pFrame, uint32_t *framestart, bool bLeft)
{
uint32_t uFrameLen = framestart[pFrame->uFrameNum+1]-framestart[pFrame->uFrameNum];
uint32_t uFilePos=0;
uint16_t uBlockLen=0, i=0, j=0;
bool bFirstLonger=false;
if (!pFrame->bHasRamps) // sanity check
return NULL;
if (pFrame->uXSize != 32) // only used in that case
return NULL;
if (!bLeft) { // get first two pixels for right side ramps
pFrame->addPixel(pFileBuf[framestart[pFrame->uFrameNum]]);
pFrame->addPixel(pFileBuf[framestart[pFrame->uFrameNum]+1]);
}
// do all the ramp blocks
for (i=0; i<(uFrameLen == c_2RampSize ? 15 : 7); i++) {
uBlockLen = (bLeft ? (c_RampOffsetLeft[i+1] - c_RampOffsetLeft[i]) : (c_RampOffsetRight[i+1] - c_RampOffsetRight[i]));
uFilePos = framestart[pFrame->uFrameNum] + (bLeft ? c_RampOffsetLeft[i] : c_RampOffsetRight[i]) + 2;
bFirstLonger = (i>(bLeft ? 7 : 6));
if (bLeft) {
// OK, first line, starting with transparency
for (j=0; j<pFrame->uXSize - uBlockLen/2 + (bFirstLonger ? 0 : 2); j++)
pFrame->addPixel(TRANS_COL);
// fill it up with the pixel block
for (j=0; j<uBlockLen/2 - (bFirstLonger ? 0 : 2); j++)
pFrame->addPixel(pFileBuf[uFilePos++]);
// second line, starting again with transparency
for (j=0; j<pFrame->uXSize - uBlockLen/2 + (bFirstLonger ? 2 : 0); j++)
pFrame->addPixel(TRANS_COL);
// fill the second line with the remaining pixels
for (j=0; j<uBlockLen/2 - (bFirstLonger ? 2 : 0); j++)
pFrame->addPixel(pFileBuf[uFilePos++]);
} else {
if (pFrame->uCols != 0) // fill current line with trans (if not empty)
for (j=pFrame->uXSize - pFrame->uCols; j>0; j--)
pFrame->addPixel(TRANS_COL);
// OK, insert the first pixels into a new line
for (j=0; j<uBlockLen/2 - (bFirstLonger ? 0 : 2); j++)
pFrame->addPixel(pFileBuf[uFilePos++]);
// fill the line with transparency
for (j=0; j<pFrame->uXSize - uBlockLen/2 + (bFirstLonger ? 0 : 2); j++)
pFrame->addPixel(TRANS_COL);
// start a second line with the remaining pixels
for (j=0; j<uBlockLen/2 - (bFirstLonger ? 2 : 0); j++)
pFrame->addPixel(pFileBuf[uFilePos++]);
}
}
// now read the last 0x00 pair and fill up
uBlockLen = (uFrameLen == c_2RampSize ? 30 : 2); // one or two ramps?
uFilePos = framestart[pFrame->uFrameNum] + (bLeft ? c_RampOffsetLeft[i] : c_RampOffsetRight[i]) + 2;
// the transparency for the last (single) 0x00 pair
for (j=0; j<uBlockLen; j++)
pFrame->addPixel(TRANS_COL);
if (bLeft) { // left side only: the remaining line
for (j=0; j<pFrame->uXSize - uBlockLen; j++)
pFrame->addPixel(pFileBuf[uFilePos++]);
}
// now the rest of the file (plain)
while (uFilePos < framestart[pFrame->uFrameNum+1])
pFrame->addPixel(pFileBuf[uFilePos++]);
// the uppermost line is emtpy when 2 ramps are used
if (uFrameLen == c_2RampSize)
for (j=0; j<pFrame->uXSize; j++)
pFrame->addPixel(TRANS_COL);
return pFrame->getData();
}
/***** celGetFrameData *************************************************
* decode .cel data for given frame and xsize
* Args:
* *vpFileBuf the buffer containing the filecontent
* *palette the palette (4 bytes for each of the 257 entries)
* 256 colors are needed + 1 for alpha
* uXSize this information must be given
* uFrameNume the frame to get
* *uYSize the actual value is returned therein
* *uMaxX this can be used (if != NULL) to get the column
* of the rightmost nontransparent pixel (useable
* eg for fonts)
*
* Returns: an array containing 4 Bytes (RGBA) for each pixel
*
* ---------------------------------------------------------------
* Comments: dirty hack, started from scratch @ 2000-10-11
* cleanly rewritten during incorporation into ladiks StormLib
* status: structured hack ;)
*
* It took me approx. 6 days to understand the format basics (hex viewer)
* For this I had a little help from a dos tool ("ddecode", from
* www.cowlevel.com, binary only, no sources) which, however, gave
* me the general idea what the pictures are actually supposed to look like.
*
* The fine adjustments, however, took quite some time and a little luck.
* After I had written to various people (mickyk and ladik), which could
* not help me, but wished best luck (thanks, btw, it helped ;)), I tried
* some reverse engineering which was not succesful in the end.
*
* I then had incidentally a new idea of what could be going on @ 2002-01-23.
* It just came to my mind that I could retry some actual painting in
* reverse order (had done that before to no avail) and when looking closer
* at it I realized the "ramp" stuff. This really is the trickiest part and
* it took me some eight days to implement it without breaking the other
* parts of the code. Very odd format indeed.
*
* TODO: learn what 0x0A 0x00 means
**********************************************************************/
uint8_t * WINAPI celGetFrameData(uint8_t *pFileBuf, uint8_t *palette, uint16_t uXSize, uint16_t uFrameNum, uint16_t *uYSize, uint16_t *uMaxX)
{
FrameBuf *pFrame;
uint32_t *framestart=NULL, frames=0, uFilePos=0;
uint16_t i, tmpWord=0;
uint8_t cRead=0, *data;
memcpy(&frames, pFileBuf, sizeof(uint32_t));
uFilePos += sizeof(uint32_t);
if (pFileBuf == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (palette == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (uFrameNum > frames-1) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (uYSize == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
// in case we want to know the rightmost pixels column (usable eg. for fonts)
if (uMaxX != NULL)
*uMaxX = 0;
// get the frame offsets
framestart = new uint32_t[frames+1];
for (i=0; i<frames+1; i++) {
memcpy(&framestart[i], pFileBuf+uFilePos, sizeof(uint32_t));
uFilePos += sizeof(uint32_t);
}
/****** block size *************************************************
* depends on the image width
******************************/
double erg = rint(sqrt(pow(2, rint(log((double)(framestart[uFrameNum+1] - framestart[uFrameNum])) / log(2.0)))));
pFrame = new FrameBuf(palette, uFrameNum, uXSize, uYSize, uMaxX, max((uint16_t)min((int)erg, 0x7F), uXSize));
/****** ramp detection -- AFAIK only needed for 32x32 tiles ********
* here I use hard coded constants because this is the only simple
* way to get the detection done; plus this stuff is only to be
* found in such 32x32 (tile) files and so wont hurt anyone ;)
******************************************************************/
uint32_t uFrameLen = framestart[uFrameNum+1] - framestart[uFrameNum];
if ((uXSize == 32) && ((uFrameLen == c_2RampSize) || (uFrameLen == c_1RampSize))) {
// use the static arrays for the check
for (i=0; i<(uFrameLen == c_2RampSize ? 16 : 8); i++) {
memcpy(&tmpWord, pFileBuf+framestart[uFrameNum]+c_RampOffsetLeft[i], sizeof(uint16_t));
if (tmpWord != 0)
break;
}
bool bRampsLeft = pFrame->bHasRamps = (i==(uFrameLen == c_2RampSize ? 16 : 8));
if (!pFrame->bHasRamps) { // only one can apply
for (i=0; i<(uFrameLen == c_2RampSize ? 16 : 8); i++) {
memcpy(&tmpWord, pFileBuf+framestart[uFrameNum]+c_RampOffsetRight[i], sizeof(uint16_t));
if (tmpWord != 0)
break;
}
pFrame->bHasRamps = (i==(uFrameLen == c_2RampSize ? 16 : 8)); // bRampsLeft stays false in this case
}
if (pFrame->bHasRamps) { // decode ramps and be off (if appropriate)
data = celDecodeRamps(pFileBuf, pFrame, framestart, bRampsLeft);
delete pFrame;
delete[] framestart;
return data;
}
}
/*********** block detection ***************************************
* 0x0A as start byte seems to be sufficient (though I still dunno
* what the trailing 10 bytes mean); in any other case we act as if
* blocks were to be used and check afterwards if the image looks
* OK (that is, the last line has no pixels in it)
******************************************************************/
cRead = pFileBuf[framestart[uFrameNum]];
if (cRead == 0x0A) // sufficient
pFrame->bHasBlocks = true;
// if width == 32 && framelen == 32*32, assume plain
else if ((uXSize != 32) || (uFrameLen != 32*32)) { // check needed
uFilePos=framestart[uFrameNum];
i=0;
// rush through the frame
while (uFilePos < framestart[uFrameNum+1]) {
cRead = pFileBuf[uFilePos++];
// transparency blocks
while (cRead > 0x7F) {
i += 256-cRead;
i %= uXSize;
if (uFilePos < framestart[uFrameNum+1])
cRead = pFileBuf[uFilePos++];
else
cRead = 0;
}
// colored pixel block
if (uFilePos < framestart[uFrameNum+1]) {
if (cRead < pFrame->uMaxBlock + 1) {
i+=cRead;
i%=uXSize;
uFilePos+=cRead;
} else {
// when the value is out of valid blockrange
i=1; // trigger error (1%uXSize != 0)
break;
}
}
}
if (i%uXSize == 0) // looks as if we got it right
pFrame->bHasBlocks=true;
}
if (pFrame->bHasBlocks) { // use block decoder if appropriate
data = celDecodeBlocks(pFileBuf, pFrame, framestart);
delete pFrame;
delete[] framestart;
return data;
}
// plain mode (#3), read each color index and write the pixel
uFilePos=framestart[uFrameNum];
while (uFilePos < framestart[uFrameNum+1])
pFrame->addPixel(pFileBuf[uFilePos++]);
// cleanup, return image data and height
data = pFrame->getData();
delete pFrame;
delete[] framestart;
return data;
}
uint16_t WINAPI cl2GetFrameCount(uint8_t *pFileBuf)
{
uint32_t tmp;
memcpy(&tmp, pFileBuf, sizeof(uint32_t));
memcpy(&tmp, pFileBuf+tmp, sizeof(uint32_t));
return (uint16_t)tmp;
}
/***** cl2GetDirData ***************************************************
* decodes all frames of a .cl2 for given direction and xsize
* Args:
* *pFileBuf the buffer containing the filecontent
* *palette the palette (4 bytes for each of the 257 entries)
* 256 colors are needed + 1 for alpha
* uXSize this information must be given
* uDirNum the direction to get the frames from
* *uYSize the actual height is returned herein
*
* Returns: <frames> arrays containing 4 Bytes (RGBA) for each pixel
* where <frames> is read at runtime and handed back via *uFrames
*
* ---------------------------------------------------------------
* Comments: dirty hack, started from scratch @ 2000-10-12
*
* The format basics are similar to .cel, with the main difference
* that the values read have reverse interpretation. In .cel a value
* greater than 0x7F means transparency, while in .cl2 this means
* color and vice-versa. .cl2 has the additional understanding
* of blocks of the same color (0x80 .. 0xBF) where the one color is
* written multiple times.
*
* .cl2 only uses the block scheme, so there is no detection
* necessary in order to get it right. The only thing still unknown
* is that 0x0A 0x00 stuff...
*
* TODO: learn what 0x0A 0x00 means
***********************************************************************/
BYTE ** WINAPI cl2GetDirData(BYTE *pFileBuf, BYTE *palette, WORD uXSize, WORD uDirNum, WORD *uYSize)
{
FrameBuf *pFrame;
uint32_t frames=0, *framestart=NULL, uFilePos=0;
uint16_t i, fc;
uint8_t cRead=0, **data=NULL;
if (pFileBuf == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (palette == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (uDirNum > 7) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (uYSize == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
// get direction offsets
uint32_t dirstart[8];
for (i=0; i<8; i++) {
memcpy(&dirstart[i], pFileBuf+uFilePos, sizeof(uint32_t));
uFilePos += sizeof(uint32_t);
}
uFilePos = dirstart[uDirNum];
memcpy(&frames, pFileBuf+uFilePos, sizeof(uint32_t));
uFilePos += sizeof(uint32_t);
data = new uint8_t*[frames];
// get frame offsets
framestart = new uint32_t[frames+1];
for (i=0; i<frames+1; i++) {
memcpy(&framestart[i], pFileBuf+uFilePos, sizeof(uint32_t));
uFilePos += sizeof(uint32_t);
}
// get frame data
for (fc=0; fc<frames; fc++) {
pFrame = new FrameBuf(palette, 0, uXSize, uYSize, NULL, 0);
uFilePos = dirstart[uDirNum] + framestart[fc];
while (uFilePos < dirstart[uDirNum] + framestart[fc+1]) {
cRead = pFileBuf[uFilePos++];
if (cRead < 0x80) { // transparency
// TODO: what is this 0x0A 0x00 stuff all about?
if ((cRead == 0x0A) && (pFileBuf[uFilePos] == 0) && (uFilePos == dirstart[uDirNum] + framestart[fc] + 1))
uFilePos += 9; // ignore the 9 bytes after 0x0A 0x00 at the framestart
else
for (i=0; i<cRead; i++)
pFrame->addPixel(TRANS_COL);
} else if (cRead < 0xC0) {
// read the next byte and write it <0xBF - cRead> times
for (i=0; i<0xBF - cRead; i++)
pFrame->addPixel(pFileBuf[uFilePos]);
++uFilePos;
} else // cRead > 0xBF
// read a block of the given size and write it
for (i=0; i < 256-cRead; i++)
pFrame->addPixel(pFileBuf[uFilePos++]);
}
// got the frame data, save it
data[fc] = pFrame->getData();
delete pFrame;
}
delete[] framestart;
return data;
}
/****** pcxGetData *****************************************************
* decodes pcx files (256 color, as in diablo mpq)
* Args:
* *pFileBuf the buffer containing the filecontent
* uFileSize the size of the file buffer
* uTransColor the palette entry to be transparent
* *uXSize the actual width is returned herein
* *uYSize the actual height is returned herein
*
* Returns: an array containing 4 Bytes (RGBA) for each pixel
*
* ---------------------------------------------------------------
* Comments: format info and pseudocode taken from:
* Klaus Holtorf, "Das Handbuch der Grafikformate"
* ISBN 3-7723-6393-8
***********************************************************************/
BYTE * WINAPI pcxGetData(BYTE *pFileBuf, DWORD uFileSize, BYTE uTransColor, WORD *uXSize, WORD *uYSize)
{
uint32_t uFilePos=0;
uint32_t uDataRead=0; // potentially big! (logo.pcx: 550 * 216 * 15 = 1,782,000)
uint16_t i=0;
uint8_t *data, *palette;
uint8_t uColorNum=0, uCount=0;
struct pcx_header_t {
uint8_t id;
uint8_t version;
uint8_t compressed;
uint8_t bpp;
uint16_t x0;
uint16_t y0;
uint16_t x1;
uint16_t y1;
uint16_t xdpi;
uint16_t ydpi;
uint8_t pal[16][3];
uint8_t reserved;
uint8_t layers;
uint16_t rowbytes;
uint16_t colortype;
uint8_t pad[58];
} pcxHeader;
if (pFileBuf == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (uXSize == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
if (uYSize == NULL) {
SetLastError(ERROR_INVALID_PARAMETER);
return NULL;
}
// get image information
memcpy(&pcxHeader, pFileBuf, sizeof(struct pcx_header_t));
*uXSize = (pcxHeader.x1 - pcxHeader.x0 + 1);
*uYSize = (pcxHeader.y1 - pcxHeader.y0 + 1);
if ((pcxHeader.version != 2) && (pcxHeader.version != 5)) {
cerr << "cannot handle pcx v" << pcxHeader.version << "\n";
return NULL;
}
// get palette
palette = new uint8_t[256*4];
if (pFileBuf[uFileSize - 768 - 1] != 0x0C) {
cerr << "palette error at " << uFileSize - 768 - 1 << "\n";
return NULL;
}
for (i=0; i<256; i++) {
memcpy(palette+i*4, pFileBuf+uFileSize-768+i*3, 3*sizeof(uint8_t));
palette[i*4+3] = 0xFF;
}
memset(palette+uTransColor*4, 0, 4*sizeof(uint8_t)); // transparent black
// start right after the header
uFilePos = sizeof(struct pcx_header_t);
data = new uint8_t[*uXSize * *uYSize * 4];
while (uDataRead < (uint32_t)(*uXSize * *uYSize)) {
// decompress
uColorNum = pFileBuf[uFilePos++];
if ((pcxHeader.compressed) && (uColorNum > 0xBF)) {
uCount = (uColorNum & 0x3F);
uColorNum = pFileBuf[uFilePos++];
} else
uCount = 1;
// draw count pixels with color val
for (i=0; i<uCount; i++)
memcpy(data+(uDataRead++)*4, palette+uColorNum*4, 4*sizeof(uint8_t));
}
// cleanup
delete[] palette;
return data;
}

1074
contrib/vmap_extractor_v2/stormlib/SCommon.cpp
View file

File diff suppressed because it is too large Load diff

88
contrib/vmap_extractor_v2/stormlib/SCommon.h
View file

@ -1,88 +0,0 @@
/*****************************************************************************/
/* SCommon.h Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Common functions for encryption/decryption from Storm.dll. Included by */
/* SFile*** functions, do not include and do not use this file directly */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 24.03.03 1.00 Lad The first version of SFileCommon.h */
/* 12.06.04 1.00 Lad Renamed to SCommon.h */
/*****************************************************************************/
#ifndef __SCOMMON_H__
#define __SCOMMON_H__
//-----------------------------------------------------------------------------
// StormLib private defines
#define SFILE_TYPE_DATA 0 // Process the file as data file
#define SFILE_TYPE_WAVE 1 // Process the file as WAVe file
//-----------------------------------------------------------------------------
// External variables
extern TMPQArchive * pFirstOpen;
extern LCID lcLocale;
//-----------------------------------------------------------------------------
// Encryption and decryption functions
int PrepareStormBuffer();
void EncryptHashTable(DWORD * pdwTable, BYTE * pbKey, DWORD dwLength);
void DecryptHashTable(DWORD * pdwTable, BYTE * pbKey, DWORD dwLength);
TMPQHash * FindFreeHashEntry(TMPQArchive * ha, const char * szFileName);
void EncryptBlockTable(DWORD * pdwTable, BYTE * pbKey, DWORD dwLength);
void DecryptBlockTable(DWORD * pdwTable, BYTE * pbKey, DWORD dwLength);
DWORD DetectFileSeed(DWORD * block, DWORD decrypted);
DWORD DetectFileSeed2(DWORD * block, UINT nDwords, ...);
void EncryptMPQBlock(DWORD * pdwBlock, DWORD dwLength, DWORD dwSeed1);
void DecryptMPQBlock(DWORD * pdwBlock, DWORD dwLength, DWORD dwSeed1);
DWORD DecryptHashIndex(TMPQArchive * ha, const char * szFileName);
DWORD DecryptName1 (const char * szFileName);
DWORD DecryptName2 (const char * szFileName);
DWORD DecryptFileSeed (const char * szFileName);
TMPQHash * GetHashEntry (TMPQArchive * ha, const char * szFileName);
TMPQHash * GetHashEntryEx(TMPQArchive * ha, const char * szFileName, LCID lcLocale);
//-----------------------------------------------------------------------------
// Compression and decompression functions
int Compress_pklib (char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * pCmpType, int nCmpLevel);
int Decompress_pklib(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength);
//-----------------------------------------------------------------------------
// Checking functions
BOOL IsValidMpqHandle(TMPQArchive * ha);
BOOL IsValidFileHandle(TMPQFile * hf);
//-----------------------------------------------------------------------------
// Other functions
BOOL SFileOpenArchiveEx(const char * szMpqName, DWORD dwPriority, DWORD dwFlags, HANDLE * phMPQ, DWORD dwAccessMode = GENERIC_READ);
int AddFileToArchive(TMPQArchive * ha, HANDLE hFile, const char * szArchivedName, DWORD dwFlags, DWORD dwQuality, int nFileType, BOOL * pbReplaced);
int SetDataCompression(int nDataCompression);
int SaveMPQTables(TMPQArchive * ha);
void FreeMPQArchive(TMPQArchive *& ha);
BOOL CheckWildCard(const char * szString, const char * szWildCard);
//-----------------------------------------------------------------------------
// Listfile functions
int SListFileCreateListFile(TMPQArchive * ha);
int SListFileAddNode(TMPQArchive * ha, const char * szAddedFile);
int SListFileRemoveNode(TMPQArchive * ha, const char * szFileName);
int SListFileRenameNode(TMPQArchive * ha, const char * szOldFileName, const char * szNewFileName);
int SListFileFreeListFile(TMPQArchive * ha);
int SListFileSaveToMpq(TMPQArchive * ha);
#endif // __SCOMMON_H__

715
contrib/vmap_extractor_v2/stormlib/SCompression.cpp
View file

@ -1,715 +0,0 @@
/*****************************************************************************/
/* SCompression.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* This module serves as a bridge between StormLib code and (de)compression */
/* functions. All (de)compression calls go (and should only go) through this */
/* module. No system headers should be included in this module to prevent */
/* compile-time problems. */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 01.04.03 1.00 Lad The first version of SCompression.cpp */
/* 19.11.03 1.01 Dan Big endian handling */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
#include <string.h>
// Include functions from Pkware Data Compression Library
#include "pklib/pklib.h"
// Include functions from zlib
#ifndef __SYS_ZLIB
#include "zlib/zlib.h" // Include functions from zlib
#else
#include <zlib.h> // If zlib is available on system, use this instead
#endif
// Include functions from Huffmann compression
#include "huffman/huff.h"
// Include functions from WAVe compression
#include "wave/wave.h"
// Include functions from BZip2 compression library
#ifndef __SYS_BZLIB
#include "bzip2/bzlib.h" // Include functions from bzlib
#else
#include <bzlib.h> // If bzlib is available on system, use this instead
#endif
//-----------------------------------------------------------------------------
// Local structures
// Information about the input and output buffers for pklib
typedef struct
{
char * pInBuff; // Pointer to input data buffer
int nInPos; // Current offset in input data buffer
int nInBytes; // Number of bytes in the input buffer
char * pOutBuff; // Pointer to output data buffer
int nOutPos; // Position in the output buffer
int nMaxOut; // Maximum number of bytes in the output buffer
} TDataInfo;
// Table of compression functions
typedef int (*COMPRESS)(char *, int *, char *, int, int *, int);
typedef struct
{
unsigned long dwMask; // Compression mask
COMPRESS Compress; // Compression function
} TCompressTable;
// Table of decompression functions
typedef int (*DECOMPRESS)(char *, int *, char *, int);
typedef struct
{
unsigned long dwMask; // Decompression bit
DECOMPRESS Decompress; // Decompression function
} TDecompressTable;
/*****************************************************************************/
/* */
/* Support functions for Pkware Data Compression Library */
/* */
/*****************************************************************************/
// Function loads data from the input buffer. Used by Pklib's "implode"
// and "explode" function as user-defined callback
// Returns number of bytes loaded
//
// char * buf - Pointer to a buffer where to store loaded data
// unsigned int * size - Max. number of bytes to read
// void * param - Custom pointer, parameter of implode/explode
static unsigned int ReadInputData(char * buf, unsigned int * size, void * param)
{
TDataInfo * pInfo = (TDataInfo *)param;
unsigned int nMaxAvail = (pInfo->nInBytes - pInfo->nInPos);
unsigned int nToRead = *size;
// Check the case when not enough data available
if(nToRead > nMaxAvail)
nToRead = nMaxAvail;
// Load data and increment offsets
memcpy(buf, pInfo->pInBuff + pInfo->nInPos, nToRead);
pInfo->nInPos += nToRead;
return nToRead;
}
// Function for store output data. Used by Pklib's "implode" and "explode"
// as user-defined callback
//
// char * buf - Pointer to data to be written
// unsigned int * size - Number of bytes to write
// void * param - Custom pointer, parameter of implode/explode
static void WriteOutputData(char * buf, unsigned int * size, void * param)
{
TDataInfo * pInfo = (TDataInfo *)param;
unsigned int nMaxWrite = (pInfo->nMaxOut - pInfo->nOutPos);
unsigned int nToWrite = *size;
// Check the case when not enough space in the output buffer
if(nToWrite > nMaxWrite)
nToWrite = nMaxWrite;
// Write output data and increments offsets
memcpy(pInfo->pOutBuff + pInfo->nOutPos, buf, nToWrite);
pInfo->nOutPos += nToWrite;
}
/*****************************************************************************/
/* */
/* "80" is IMA ADPCM stereo (de)compression */
/* "40" is IMA ADPCM mono (de)compression */
/* */
/*****************************************************************************/
int Compress_wave_mono(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * pCmpType, int nCmpLevel)
{
// Prepare the compression level for the next compression
// (After us, the Huffmann compression will be called)
if(0 < nCmpLevel && nCmpLevel <= 2)
{
nCmpLevel = 4;
*pCmpType = 6;
}
else if(nCmpLevel == 3)
{
nCmpLevel = 6;
*pCmpType = 8;
}
else
{
nCmpLevel = 5;
*pCmpType = 7;
}
*pdwOutLength = CompressWave((unsigned char *)pbOutBuffer, *pdwOutLength, (short *)pbInBuffer, dwInLength, 1, nCmpLevel);
return 0;
}
int Decompress_wave_mono(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
*pdwOutLength = DecompressWave((unsigned char *)pbOutBuffer, *pdwOutLength, (unsigned char *)pbInBuffer, dwInLength, 1);
return 1;
}
int Compress_wave_stereo(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * pCmpType, int nCmpLevel)
{
// Prepare the compression type for the next compression
// (After us, the Huffmann compression will be called)
if(0 < nCmpLevel && nCmpLevel <= 2)
{
nCmpLevel = 4;
*pCmpType = 6;
}
else if(nCmpLevel == 3)
{
nCmpLevel = 6;
*pCmpType = 8;
}
else
{
nCmpLevel = 5;
*pCmpType = 7;
}
*pdwOutLength = CompressWave((unsigned char *)pbOutBuffer, *pdwOutLength, (short *)pbInBuffer, dwInLength, 2, nCmpLevel);
return 0;
}
int Decompress_wave_stereo(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
*pdwOutLength = DecompressWave((unsigned char *)pbOutBuffer, *pdwOutLength, (unsigned char *)pbInBuffer, dwInLength, 2);
return 1;
}
/*****************************************************************************/
/* */
/* The "01" (de)compression is the Huffman (de)compression */
/* */
/*****************************************************************************/
// 1500F4C0
int Compress_huff(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * pCmpType, int /* nCmpLevel */)
{
THuffmannTree ht; // Huffmann tree for compression
TOutputStream os; // Output stream
// Initialize output stream
os.pbOutBuffer = (unsigned char *)pbOutBuffer;
os.dwOutSize = *pdwOutLength;
os.pbOutPos = (unsigned char *)pbOutBuffer;
os.dwBitBuff = 0;
os.nBits = 0;
// Initialize the Huffmann tree for compression
ht.InitTree(true);
*pdwOutLength = ht.DoCompression(&os, (unsigned char *)pbInBuffer, dwInLength, *pCmpType);
// The following code is not necessary to run, because it has no
// effect on the output data. It only clears the huffmann tree, but when
// the tree is on the stack, who cares ?
// ht.UninitTree();
return 0;
}
// 1500F5F0
int Decompress_huff(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int /* dwInLength */)
{
THuffmannTree ht;
TInputStream is;
// Initialize input stream
// is.pbInBuffer = (unsigned char *)pbInBuffer;
is.dwBitBuff = BSWAP_INT32_UNSIGNED(*(unsigned long *)pbInBuffer);
pbInBuffer += sizeof(unsigned long);
is.pbInBuffer = (unsigned char *)pbInBuffer;
is.nBits = 32;
// Initialize the Huffmann tree for compression
ht.InitTree(false);
*pdwOutLength = ht.DoDecompression((unsigned char *)pbOutBuffer, *pdwOutLength, &is);
// The following code is not necessary to run, because it has no
// effect on the output data. It only clears the huffmann tree, but when
// the tree is on the stack, who cares ?
// ht.UninitTree();
return 0;
}
/*****************************************************************************/
/* */
/* The "02" (de)compression is the ZLIB (de)compression */
/* */
/*****************************************************************************/
int Compress_zlib(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * /* pCmpType */, int /* nCmpLevel */)
{
z_stream z; // Stream information for zlib
int nResult;
// Fill the stream structure for zlib
z.next_in = (Bytef *)pbInBuffer;
z.avail_in = (uInt)dwInLength;
z.total_in = dwInLength;
z.next_out = (Bytef *)pbOutBuffer;
z.avail_out = *pdwOutLength;
z.total_out = 0;
z.zalloc = NULL;
z.zfree = NULL;
// Initialize the compression structure. Storm.dll uses zlib version 1.1.3
*pdwOutLength = 0;
if((nResult = deflateInit(&z, Z_DEFAULT_COMPRESSION)) == 0)
{
// Call zlib to compress the data
nResult = deflate(&z, Z_FINISH);
if(nResult == Z_OK || nResult == Z_STREAM_END)
*pdwOutLength = z.total_out;
deflateEnd(&z);
}
return nResult;
}
int Decompress_zlib(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
z_stream z; // Stream information for zlib
int nResult;
// Fill the stream structure for zlib
z.next_in = (Bytef *)pbInBuffer;
z.avail_in = (uInt)dwInLength;
z.total_in = dwInLength;
z.next_out = (Bytef *)pbOutBuffer;
z.avail_out = *pdwOutLength;
z.total_out = 0;
z.zalloc = NULL;
z.zfree = NULL;
// Initialize the decompression structure. Storm.dll uses zlib version 1.1.3
if((nResult = inflateInit(&z)) == 0)
{
// Call zlib to decompress the data
nResult = inflate(&z, Z_FINISH);
*pdwOutLength = z.total_out;
inflateEnd(&z);
}
return nResult;
}
/*****************************************************************************/
/* */
/* The "08" (de)compression is the Pkware DCL (de)compression */
/* */
/*****************************************************************************/
int Compress_pklib(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * pCmpType, int /* nCmpLevel */)
{
TDataInfo Info; // Data information
char * work_buf = ALLOCMEM(char, CMP_BUFFER_SIZE);// Pklib's work buffer
unsigned int dict_size; // Dictionary size
unsigned int ctype; // Compression type
// Fill data information structure
Info.pInBuff = pbInBuffer;
Info.nInPos = 0;
Info.nInBytes = dwInLength;
Info.pOutBuff = pbOutBuffer;
Info.nOutPos = 0;
Info.nMaxOut = *pdwOutLength;
// Set the compression type and dictionary size
ctype = (*pCmpType == 2) ? CMP_ASCII : CMP_BINARY;
if (dwInLength < 0x600)
dict_size = 0x400;
else if(0x600 <= dwInLength && dwInLength < 0xC00)
dict_size = 0x800;
else
dict_size = 0x1000;
// Do the compression
implode(ReadInputData, WriteOutputData, work_buf, &Info, &ctype, &dict_size);
*pdwOutLength = Info.nOutPos;
FREEMEM(work_buf);
return 0;
}
int Decompress_pklib(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
TDataInfo Info; // Data information
char * work_buf = ALLOCMEM(char, EXP_BUFFER_SIZE);// Pklib's work buffer
// Fill data information structure
Info.pInBuff = pbInBuffer;
Info.nInPos = 0;
Info.nInBytes = dwInLength;
Info.pOutBuff = pbOutBuffer;
Info.nOutPos = 0;
Info.nMaxOut = *pdwOutLength;
// Do the decompression
explode(ReadInputData, WriteOutputData, work_buf, &Info);
// Fix: If PKLIB is unable to decompress the data, they are uncompressed
if(Info.nOutPos == 0)
{
Info.nOutPos = min(*pdwOutLength, dwInLength);
memcpy(pbOutBuffer, pbInBuffer, Info.nOutPos);
}
*pdwOutLength = Info.nOutPos;
FREEMEM(work_buf);
return 0;
}
/*****************************************************************************/
/* */
/* The "10" (de)compression is the Bzip2 (de)compression */
/* */
/*****************************************************************************/
int Compress_bzip2(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int * pCmpType, int nCmpLevel)
{
bz_stream strm;
int blockSize100k;
int workFactor = 30;
// Keep compiler happy
nCmpLevel = nCmpLevel;
// Initialize the BZLIB compression
strm.bzalloc = NULL;
strm.bzfree = NULL;
// Adjust the block size
blockSize100k = *pCmpType;
if(blockSize100k < 1 || blockSize100k > 9)
blockSize100k = 9;
// Blizzard uses 9 as blockSize100k, (0 as workFactor)
if(BZ2_bzCompressInit(&strm, blockSize100k, 0, workFactor) == 0)
{
strm.next_in = pbInBuffer;
strm.avail_in = dwInLength;
strm.next_out = pbOutBuffer;
strm.avail_out = *pdwOutLength;
// Perform the compression
while(BZ2_bzCompress(&strm, (strm.avail_in != 0) ? BZ_RUN : BZ_FINISH) != BZ_STREAM_END);
// Put the stream into idle state
BZ2_bzCompressEnd(&strm);
*pdwOutLength = strm.total_out_lo32;
}
else
{
*pdwOutLength = 0;
}
return 0;
}
int Decompress_bzip2(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
bz_stream strm;
// Initialize the BZLIB decompression
strm.bzalloc = NULL;
strm.bzfree = NULL;
if(BZ2_bzDecompressInit(&strm, 0, 0) == 0)
{
strm.next_in = pbInBuffer;
strm.avail_in = dwInLength;
strm.next_out = pbOutBuffer;
strm.avail_out = *pdwOutLength;
// Perform the decompression
while(BZ2_bzDecompress(&strm) != BZ_STREAM_END);
// Put the stream into idle state
BZ2_bzDecompressEnd(&strm);
*pdwOutLength = strm.total_out_lo32;
}
else
{
// Set zero output length
*pdwOutLength = 0;
}
return 0;
}
/*****************************************************************************/
/* */
/* SCompCompress */
/* */
/*****************************************************************************/
// This table contains compress functions which can be applied to
// uncompressed blocks. Each bit set means the corresponding
// compression method/function must be applied.
//
// WAVes compression Data compression
// ------------------ -------------------
// 1st block - 0x08 0x08 (D, HF, W2, SC, D2)
// Rest blocks - 0x81 0x02 (W3)
static TCompressTable cmp_table[] =
{
{MPQ_COMPRESSION_WAVE_MONO, Compress_wave_mono}, // IMA ADPCM mono compression
{MPQ_COMPRESSION_WAVE_STEREO, Compress_wave_stereo}, // IMA ADPCM stereo compression
{MPQ_COMPRESSION_HUFFMANN, Compress_huff}, // Huffmann compression
{MPQ_COMPRESSION_ZLIB, Compress_zlib}, // Compression with the "zlib" library
{MPQ_COMPRESSION_PKWARE, Compress_pklib}, // Compression with Pkware DCL
{MPQ_COMPRESSION_BZIP2, Compress_bzip2} // Compression Bzip2 library
};
int WINAPI SCompCompress(char * pbCompressed, int * pdwOutLength, char * pbUncompressed, int dwInLength,
int uCompressions, int nCmpType, int nCmpLevel)
{
char * pbTempBuff = NULL; // Temporary storage for decompressed data
char * pbOutput = pbCompressed; // Current output buffer
char * pbInput; // Current input buffer
int uCompressions2;
int dwCompressCount = 0;
int dwDoneCount = 0;
int dwOutSize = 0;
int dwInSize = dwInLength;
int dwEntries = (sizeof(cmp_table) / sizeof(TCompressTable));
int nResult = 1;
int i;
// Check for valid parameters
if(!pdwOutLength || *pdwOutLength < dwInLength || !pbCompressed || !pbUncompressed)
{
SetLastError(ERROR_INVALID_PARAMETER);
return 0;
}
// Count the compressions
for(i = 0, uCompressions2 = uCompressions; i < dwEntries; i++)
{
if(uCompressions & cmp_table[i].dwMask)
dwCompressCount++;
uCompressions2 &= ~cmp_table[i].dwMask;
}
// If a compression remains, do nothing
if(uCompressions2 != 0)
{
SetLastError(ERROR_INVALID_PARAMETER);
return 0;
}
// If more that one compression, allocate intermediate buffer
if(dwCompressCount >= 2)
pbTempBuff = ALLOCMEM(char, *pdwOutLength + 1);
// Perform the compressions
pbInput = pbUncompressed;
dwInSize = dwInLength;
for(i = 0, uCompressions2 = uCompressions; i < dwEntries; i++)
{
if(uCompressions2 & cmp_table[i].dwMask)
{
// Set the right output buffer
dwCompressCount--;
pbOutput = (dwCompressCount & 1) ? pbTempBuff : pbCompressed;
// Perform the partial compression
dwOutSize = *pdwOutLength - 1;
cmp_table[i].Compress(pbOutput + 1, &dwOutSize, pbInput, dwInSize, &nCmpType, nCmpLevel);
if(dwOutSize == 0)
{
SetLastError(ERROR_GEN_FAILURE);
*pdwOutLength = 0;
nResult = 0;
break;
}
// If the compression failed, copy the block instead
if(dwOutSize >= dwInSize - 1)
{
if(dwDoneCount > 0)
pbOutput++;
memcpy(pbOutput, pbInput, dwInSize);
pbInput = pbOutput;
uCompressions &= ~cmp_table[i].dwMask;
dwOutSize = dwInSize;
}
else
{
pbInput = pbOutput + 1;
dwInSize = dwOutSize;
dwDoneCount++;
}
}
}
// Copy the compressed data to the correct output buffer
if(nResult != 0)
{
if(uCompressions && (dwInSize + 1) < *pdwOutLength)
{
if(pbOutput != pbCompressed && pbOutput != pbCompressed + 1)
memcpy(pbCompressed, pbOutput, dwInSize);
*pbCompressed = (char)uCompressions;
*pdwOutLength = dwInSize + 1;
}
else
{
memmove(pbCompressed, pbUncompressed, dwInSize);
*pdwOutLength = dwInSize;
}
}
// Cleanup and return
if(pbTempBuff != NULL)
FREEMEM(pbTempBuff);
return nResult;
}
/*****************************************************************************/
/* */
/* SCompDecompress */
/* */
/*****************************************************************************/
// This table contains decompress functions which can be applied to
// uncompressed blocks. The compression mask is stored in the first byte
// of compressed block
static TDecompressTable dcmp_table[] =
{
{MPQ_COMPRESSION_BZIP2, Decompress_bzip2}, // Decompression with Bzip2 library
{MPQ_COMPRESSION_PKWARE, Decompress_pklib}, // Decompression with Pkware Data Compression Library
{MPQ_COMPRESSION_ZLIB, Decompress_zlib}, // Decompression with the "zlib" library
{MPQ_COMPRESSION_HUFFMANN, Decompress_huff}, // Huffmann decompression
{MPQ_COMPRESSION_WAVE_STEREO, Decompress_wave_stereo}, // IMA ADPCM stereo decompression
{MPQ_COMPRESSION_WAVE_MONO, Decompress_wave_mono} // IMA ADPCM mono decompression
};
int WINAPI SCompDecompress(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength)
{
char * pbTempBuff = NULL; // Temporary storage for decompressed data
char * pbWorkBuff = NULL; // Where to store decompressed data
int dwOutLength = *pdwOutLength; // For storage number of output bytes
unsigned fDecompressions1; // Decompressions applied to the block
unsigned fDecompressions2; // Just another copy of decompressions applied to the block
int dwCount = 0; // Counter for every use
int dwEntries = (sizeof(dcmp_table) / sizeof(TDecompressTable));
int nResult = 1;
int i;
// If the input length is the same as output, do nothing.
if(dwInLength == dwOutLength)
{
if(pbInBuffer == pbOutBuffer)
return 1;
memcpy(pbOutBuffer, pbInBuffer, dwInLength);
*pdwOutLength = dwInLength;
return 1;
}
// Get applied compression types and decrement data length
fDecompressions1 = fDecompressions2 = (unsigned char)*pbInBuffer++;
dwInLength--;
// Search decompression table type and get all types of compression
for(i = 0; i < dwEntries; i++)
{
// We have to apply this decompression ?
if(fDecompressions1 & dcmp_table[i].dwMask)
dwCount++;
// Clear this flag from temporary variable.
fDecompressions2 &= ~dcmp_table[i].dwMask;
}
// Check if there is some method unhandled
// (E.g. compressed by future versions)
if(fDecompressions2 != 0)
{
SetLastError(ERROR_INVALID_PARAMETER);
return 0;
}
// If there is more than only one compression, we have to allocate extra buffer
if(dwCount >= 2)
pbTempBuff = ALLOCMEM(char, dwOutLength);
// Apply all decompressions
for(i = 0, dwCount = 0; i < dwEntries; i++)
{
// If not used this kind of compression, skip the loop
if(fDecompressions1 & dcmp_table[i].dwMask)
{
// If odd case, use target buffer for output, otherwise use allocated tempbuffer
pbWorkBuff = (dwCount++ & 1) ? pbTempBuff : pbOutBuffer;
dwOutLength = *pdwOutLength;
// Decompress buffer using corresponding function
dcmp_table[i].Decompress(pbWorkBuff, &dwOutLength, pbInBuffer, dwInLength);
if(dwOutLength == 0)
{
SetLastError(ERROR_GEN_FAILURE);
nResult = 0;
break;
}
// Move output length to src length for next compression
dwInLength = dwOutLength;
pbInBuffer = pbWorkBuff;
}
}
// If output buffer is not the same like target buffer, we have to copy data
if(nResult != 0)
{
if(pbWorkBuff != pbOutBuffer)
memcpy(pbOutBuffer, pbInBuffer, dwOutLength);
}
// Delete temporary buffer, if necessary
if(pbTempBuff != NULL)
FREEMEM(pbTempBuff);
*pdwOutLength = dwOutLength;
return nResult;
}
/*****************************************************************************/
/* */
/* SCompSetDataCompression */
/* */
/*****************************************************************************/
int WINAPI SCompSetDataCompression(int nDataCompression)
{
int nValidMask = (MPQ_COMPRESSION_ZLIB | MPQ_COMPRESSION_PKWARE | MPQ_COMPRESSION_BZIP2);
if((nDataCompression & nValidMask) != nDataCompression)
{
SetLastError(ERROR_INVALID_PARAMETER);
return FALSE;
}
SetDataCompression(nDataCompression);
return TRUE;
}

691
contrib/vmap_extractor_v2/stormlib/SFileCompactArchive.cpp
View file

@ -1,691 +0,0 @@
/*****************************************************************************/
/* SFileCompactArchive.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Archive compacting function */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 14.04.03 1.00 Lad Splitted from SFileCreateArchiveEx.cpp */
/* 19.11.03 1.01 Dan Big endian handling */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
/*****************************************************************************/
/* Local structures */
/*****************************************************************************/
/*****************************************************************************/
/* Local variables */
/*****************************************************************************/
static COMPACTCB CompactCB = NULL;
static void * lpUserData = NULL;
/*****************************************************************************/
/* Local functions */
/*****************************************************************************/
// Creates a copy of hash table
static TMPQHash * CopyHashTable(TMPQArchive * ha)
{
TMPQHash * pHashTableCopy = ALLOCMEM(TMPQHash, ha->pHeader->dwHashTableSize);
if(pHashTableCopy != NULL)
memcpy(pHashTableCopy, ha->pHashTable, sizeof(TMPQHash) * ha->pHeader->dwHashTableSize);
return pHashTableCopy;
}
// TODO: Test for archives > 4GB
static int CheckIfAllFilesKnown(TMPQArchive * ha, const char * szListFile, DWORD * pFileSeeds)
{
TMPQHash * pHashTableCopy = NULL; // Copy of the hash table
TMPQHash * pHash;
TMPQHash * pHashEnd = NULL; // End of the hash table
DWORD dwFileCount = 0;
int nError = ERROR_SUCCESS;
// First of all, create a copy of hash table
if(nError == ERROR_SUCCESS)
{
if((pHashTableCopy = CopyHashTable(ha)) == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
pHashEnd = pHashTableCopy + ha->pHeader->dwHashTableSize;
// Notify the user
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_CHECKING_FILES, 0, ha->pHeader->dwBlockTableSize);
}
// Now check all the files from the filelist
if(nError == ERROR_SUCCESS)
{
SFILE_FIND_DATA wf;
HANDLE hFind = SFileFindFirstFile((HANDLE)ha, "*", &wf, szListFile);
BOOL bResult = TRUE;
// Do while some files have been found
while(hFind != NULL && bResult)
{
TMPQHash * pHash = GetHashEntry(ha, wf.cFileName);
// If the hash table entry has been found, find it's position
// in the hash table copy
if(pHash != NULL)
{
pHash = pHashTableCopy + (pHash - ha->pHashTable);
if(pHash->dwName1 != (DWORD)-1 && pHash->dwName2 != (DWORD)-1)
{
TMPQBlock * pBlock = ha->pBlockTable + pHash->dwBlockIndex;
DWORD dwSeed = 0;
// Resolve the file seed. Use plain file name for it
if(pBlock->dwFlags & MPQ_FILE_ENCRYPTED)
{
char * szFileName = strrchr(wf.cFileName, '\\');
if(szFileName == NULL)
szFileName = wf.cFileName;
else
szFileName++;
dwSeed = DecryptFileSeed(szFileName);
if(pBlock->dwFlags & MPQ_FILE_FIXSEED)
dwSeed = (dwSeed + pBlock->dwFilePos) ^ pBlock->dwFSize;
}
pFileSeeds[pHash->dwBlockIndex] = dwSeed;
pHash->dwName1 = 0xFFFFFFFF;
pHash->dwName2 = 0xFFFFFFFF;
pHash->lcLocale = 0xFFFF;
pHash->wPlatform = 0xFFFF;
pHash->dwBlockIndex = 0xFFFFFFFF;
}
}
// Notify the user
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_CHECKING_FILES, ++dwFileCount, ha->pHeader->dwBlockTableSize);
// Find the next file in the archive
bResult = SFileFindNextFile(hFind, &wf);
}
if(hFind != NULL)
SFileFindClose(hFind);
}
// When the filelist checking is complete, parse the hash table copy and find the
if(nError == ERROR_SUCCESS)
{
// Notify the user about checking hash table
dwFileCount = 0;
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_CHECKING_HASH_TABLE, dwFileCount, ha->pHeader->dwBlockTableSize);
for(pHash = pHashTableCopy; pHash < pHashEnd; pHash++)
{
// If there is an unresolved entry, try to detect its seed. If it fails,
// we cannot complete the work
if(pHash->dwName1 != (DWORD)-1 && pHash->dwName2 != (DWORD)-1)
{
HANDLE hFile = NULL;
DWORD dwFlags = 0;
DWORD dwSeed = 0;
if(SFileOpenFileEx((HANDLE)ha, (char *)(DWORD_PTR)pHash->dwBlockIndex, 0, &hFile))
{
TMPQFile * hf = (TMPQFile *)hFile;
dwFlags = hf->pBlock->dwFlags;
dwSeed = hf->dwSeed1;
SFileCloseFile(hFile);
}
// If the file is encrypted, we have to check
// If we can apply the file decryption seed
if(dwFlags & MPQ_FILE_ENCRYPTED && dwSeed == 0)
{
nError = ERROR_CAN_NOT_COMPLETE;
break;
}
// Remember the seed
pFileSeeds[pHash->dwBlockIndex] = dwSeed;
// Notify the user
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_CHECKING_HASH_TABLE, ++dwFileCount, ha->pHeader->dwBlockTableSize);
}
}
}
// Delete the copy of hash table
if(pHashTableCopy != NULL)
FREEMEM(pHashTableCopy);
return nError;
}
// Copies all file blocks into another archive.
// TODO: Test for archives > 4GB
static int CopyMpqFileBlocks(
HANDLE hFile,
TMPQArchive * ha,
TMPQBlockEx * pBlockEx,
TMPQBlock * pBlock,
DWORD dwSeed)
{
LARGE_INTEGER FilePos = {0};
DWORD * pdwBlockPos2 = NULL; // File block positions to be written to target file
DWORD * pdwBlockPos = NULL; // File block positions (unencrypted)
BYTE * pbBlock = NULL; // Buffer for the file block
DWORD dwTransferred; // Number of bytes transferred
DWORD dwCSize = 0; // Compressed file size
DWORD dwBytes = 0; // Number of bytes
DWORD dwSeed1 = 0; // File seed used for decryption
DWORD dwSeed2 = 0; // File seed used for encryption
DWORD nBlocks = 0; // Number of file blocks
DWORD nBlock = 0; // Currently processed file block
int nError = ERROR_SUCCESS;
// When file length is zero, do nothing
if(pBlock->dwFSize == 0)
return ERROR_SUCCESS;
// Calculate number of blocks in the file
if(nError == ERROR_SUCCESS)
{
nBlocks = pBlock->dwFSize / ha->dwBlockSize;
if(pBlock->dwFSize % ha->dwBlockSize)
nBlocks++;
pbBlock = ALLOCMEM(BYTE, ha->dwBlockSize);
if(pbBlock == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Set the position to the begin of the file within archive
if(nError == ERROR_SUCCESS)
{
FilePos.HighPart = pBlockEx->wFilePosHigh;
FilePos.LowPart = pBlock->dwFilePos;
FilePos.QuadPart += ha->MpqPos.QuadPart;
if(SetFilePointer(ha->hFile, FilePos.LowPart, &FilePos.HighPart, FILE_BEGIN) != FilePos.LowPart)
nError = GetLastError();
}
// Remember the position in the destination file
if(nError == ERROR_SUCCESS)
{
FilePos.HighPart = 0;
FilePos.LowPart = SetFilePointer(hFile, 0, &FilePos.HighPart, FILE_CURRENT);
}
// Resolve decryption seeds. The 'dwSeed' parameter is the decryption
// seed for the file.
if(nError == ERROR_SUCCESS && (pBlock->dwFlags & MPQ_FILE_ENCRYPTED))
{
dwSeed1 = dwSeed;
if(pBlock->dwFlags & MPQ_FILE_FIXSEED)
dwSeed = (dwSeed1 ^ pBlock->dwFSize) - pBlock->dwFilePos;
dwSeed2 = dwSeed;
if(pBlock->dwFlags & MPQ_FILE_FIXSEED)
dwSeed2 = (dwSeed + (DWORD)(FilePos.QuadPart - ha->MpqPos.QuadPart)) ^ pBlock->dwFSize;
}
// Load the file positions from the archive and save it to the target file
// (only if the file is compressed)
if(pBlock->dwFlags & MPQ_FILE_COMPRESSED)
{
// Allocate buffers
if(nError == ERROR_SUCCESS)
{
pdwBlockPos = ALLOCMEM(DWORD, nBlocks + 2);
pdwBlockPos2 = ALLOCMEM(DWORD, nBlocks + 2);
if(pdwBlockPos == NULL || pdwBlockPos2 == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Load the block positions
if(nError == ERROR_SUCCESS)
{
dwBytes = (nBlocks + 1) * sizeof(DWORD);
if(pBlock->dwFlags & MPQ_FILE_HAS_EXTRA)
dwBytes += sizeof(DWORD);
ReadFile(ha->hFile, pdwBlockPos, dwBytes, &dwTransferred, NULL);
if(dwTransferred != dwBytes)
nError = ERROR_FILE_CORRUPT;
}
// Re-encrypt the block table positions
if(nError == ERROR_SUCCESS)
{
BSWAP_ARRAY32_UNSIGNED(pdwBlockPos, dwBytes / sizeof(DWORD));
if(pBlock->dwFlags & MPQ_FILE_ENCRYPTED)
{
DecryptMPQBlock(pdwBlockPos, dwBytes, dwSeed1 - 1);
if(pdwBlockPos[0] != dwBytes)
nError = ERROR_FILE_CORRUPT;
memcpy(pdwBlockPos2, pdwBlockPos, dwBytes);
EncryptMPQBlock(pdwBlockPos2, dwBytes, dwSeed2 - 1);
}
else
{
memcpy(pdwBlockPos2, pdwBlockPos, dwBytes);
}
BSWAP_ARRAY32_UNSIGNED(pdwBlockPos2, dwBytes / sizeof(DWORD));
}
// Write to the target file
if(nError == ERROR_SUCCESS)
{
WriteFile(hFile, pdwBlockPos2, dwBytes, &dwTransferred, NULL);
dwCSize += dwTransferred;
if(dwTransferred != dwBytes)
nError = ERROR_DISK_FULL;
}
}
// Now we have to copy all file block. We will do it without
// recompression, because re-compression is not necessary in this case
if(nError == ERROR_SUCCESS)
{
for(nBlock = 0; nBlock < nBlocks; nBlock++)
{
// Fix: The last block must not be exactly the size of one block.
dwBytes = ha->dwBlockSize;
if(nBlock == nBlocks - 1)
{
dwBytes = pBlock->dwFSize - (ha->dwBlockSize * (nBlocks - 1));
}
if(pBlock->dwFlags & MPQ_FILE_COMPRESSED)
dwBytes = pdwBlockPos[nBlock+1] - pdwBlockPos[nBlock];
// Read the file block
ReadFile(ha->hFile, pbBlock, dwBytes, &dwTransferred, NULL);
if(dwTransferred != dwBytes)
{
nError = ERROR_FILE_CORRUPT;
break;
}
// If necessary, re-encrypt the block
// Note: Recompression is not necessary here. Unlike encryption,
// the compression does not depend on the position of the file in MPQ.
if((pBlock->dwFlags & MPQ_FILE_ENCRYPTED) && dwSeed1 != dwSeed2)
{
BSWAP_ARRAY32_UNSIGNED((DWORD *)pbBlock, dwBytes/sizeof(DWORD));
DecryptMPQBlock((DWORD *)pbBlock, dwBytes, dwSeed1 + nBlock);
EncryptMPQBlock((DWORD *)pbBlock, dwBytes, dwSeed2 + nBlock);
BSWAP_ARRAY32_UNSIGNED((DWORD *)pbBlock, dwBytes/sizeof(DWORD));
}
// Now write the block back to the file
WriteFile(hFile, pbBlock, dwBytes, &dwTransferred, NULL);
dwCSize += dwTransferred;
if(dwTransferred != dwBytes)
{
nError = ERROR_DISK_FULL;
break;
}
}
}
// Copy the file extras, if any
// These extras does not seem to be encrypted, and their purpose is unknown
if(nError == ERROR_SUCCESS && (pBlock->dwFlags & MPQ_FILE_HAS_EXTRA))
{
dwBytes = pdwBlockPos[nBlocks + 1] - pdwBlockPos[nBlocks];
if(dwBytes != 0)
{
ReadFile(ha->hFile, pbBlock, dwBytes, &dwTransferred, NULL);
if(dwTransferred == dwBytes)
{
WriteFile(hFile, pbBlock, dwBytes, &dwTransferred, NULL);
dwCSize += dwTransferred;
if(dwTransferred != dwBytes)
nError = ERROR_DISK_FULL;
}
else
{
nError = ERROR_FILE_CORRUPT;
}
}
}
// Update file position in the block table
if(nError == ERROR_SUCCESS)
{
// At this point, number of bytes written should be exactly
// the same like the compressed file size. If it isn't, there's something wrong
// (maybe new archive version ?)
assert(dwCSize == pBlock->dwCSize);
// Update file pos in the block table
FilePos.QuadPart -= ha->MpqPos.QuadPart;
pBlockEx->wFilePosHigh = (USHORT)FilePos.HighPart;
pBlock->dwFilePos = FilePos.LowPart;
}
// Cleanup and return
if(pdwBlockPos2 != NULL)
FREEMEM(pdwBlockPos2);
if(pdwBlockPos != NULL)
FREEMEM(pdwBlockPos);
if(pbBlock != NULL)
FREEMEM(pbBlock);
return nError;
}
static int CopyNonMpqData(
HANDLE hSrcFile,
HANDLE hTrgFile,
LARGE_INTEGER & DataSizeToCopy)
{
LARGE_INTEGER DataSize = DataSizeToCopy;
DWORD dwTransferred;
DWORD dwToRead;
char DataBuffer[0x1000];
int nError = ERROR_SUCCESS;
while(DataSize.QuadPart > 0)
{
// Get the proper size of data
dwToRead = sizeof(DataBuffer);
if(DataSize.HighPart == 0 && DataSize.LowPart < dwToRead)
dwToRead = DataSize.LowPart;
// Read the source file
ReadFile(hSrcFile, DataBuffer, dwToRead, &dwTransferred, NULL);
if(dwTransferred != dwToRead)
{
nError = ERROR_CAN_NOT_COMPLETE;
break;
}
// Write to the target file
WriteFile(hTrgFile, DataBuffer, dwToRead, &dwTransferred, NULL);
if(dwTransferred != dwToRead)
{
nError = ERROR_DISK_FULL;
break;
}
// Decrement the number of data to be copied
DataSize.QuadPart -= dwTransferred;
}
return ERROR_SUCCESS;
}
// TODO: Test for archives > 4GB
static int CopyMpqFiles(HANDLE hFile, TMPQArchive * ha, DWORD * pFileSeeds)
{
TMPQBlockEx * pBlockEx;
TMPQBlock * pBlock;
DWORD dwSeed1;
DWORD dwIndex;
int nError = ERROR_SUCCESS;
// Walk through all files and write them to the destination MPQ archive
for(dwIndex = 0; dwIndex < ha->pHeader->dwBlockTableSize; dwIndex++)
{
pBlockEx = ha->pExtBlockTable + dwIndex;
pBlock = ha->pBlockTable + dwIndex;
dwSeed1 = pFileSeeds[dwIndex];
// Notify the caller about work
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_COMPACTING_FILES, dwIndex, ha->pHeader->dwBlockTableSize);
// if(dwIndex == 0x1B9)
// DebugBreak();
// Copy all the file blocks
// Debug: Break at (dwIndex == 5973)
if(pBlock->dwFlags & MPQ_FILE_EXISTS)
{
nError = CopyMpqFileBlocks(hFile, ha, pBlockEx, pBlock, dwSeed1);
if(nError != ERROR_SUCCESS)
break;
}
}
// Cleanup and exit
return nError;
}
/*****************************************************************************/
/* Public functions */
/*****************************************************************************/
BOOL WINAPI SFileSetCompactCallback(HANDLE /* hMPQ */, COMPACTCB aCompactCB, void * lpData)
{
CompactCB = aCompactCB;
lpUserData = lpData;
return TRUE;
}
//-----------------------------------------------------------------------------
// Archive compacting (incomplete)
// TODO: Test for archives > 4GB
BOOL WINAPI SFileCompactArchive(HANDLE hMPQ, const char * szListFile, BOOL /* bReserved */)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
HANDLE hFile = INVALID_HANDLE_VALUE;
DWORD * pFileSeeds = NULL;
char szTempFile[MAX_PATH] = "";
char * szTemp = NULL;
DWORD dwTransferred;
int nError = ERROR_SUCCESS;
// Test the valid parameters
if(!IsValidMpqHandle(ha))
nError = ERROR_INVALID_PARAMETER;
// Create the table with file seeds
if(nError == ERROR_SUCCESS)
{
if((pFileSeeds = ALLOCMEM(DWORD, ha->pHeader->dwBlockTableSize)) != NULL)
memset(pFileSeeds, 0, sizeof(DWORD) * ha->pHeader->dwBlockTableSize);
else
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// First of all, we have to check of we are able to decrypt all files.
// If not, sorry, but the archive cannot be compacted.
if(nError == ERROR_SUCCESS)
nError = CheckIfAllFilesKnown(ha, szListFile, pFileSeeds);
// Get the temporary file name and create it
if(nError == ERROR_SUCCESS)
{
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_COPYING_NON_MPQ_DATA, 0, 0);
strcpy(szTempFile, ha->szFileName);
if((szTemp = strrchr(szTempFile, '.')) != NULL)
strcpy(szTemp + 1, "mp_");
else
strcat(szTempFile, "_");
hFile = CreateFile(szTempFile, GENERIC_READ|GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, 0, NULL);
if(hFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
// Write the data before MPQ header (if any)
if(nError == ERROR_SUCCESS && ha->MpqPos.QuadPart > 0)
{
SetFilePointer(ha->hFile, 0, NULL, FILE_BEGIN);
if(ha->pShunt != NULL)
nError = CopyNonMpqData(ha->hFile, hFile, ha->ShuntPos);
else
nError = CopyNonMpqData(ha->hFile, hFile, ha->MpqPos);
}
// Write the MPQ shunt (if any)
if(nError == ERROR_SUCCESS && ha->pShunt != NULL)
{
BSWAP_TMPQSHUNT(ha->pShunt);
WriteFile(hFile, ha->pShunt, sizeof(TMPQShunt), &dwTransferred, NULL);
BSWAP_TMPQSHUNT(ha->pShunt);
if(dwTransferred != sizeof(TMPQShunt))
nError = ERROR_DISK_FULL;
}
// Write the data between MPQ shunt and the MPQ header (if any)
if(nError == ERROR_SUCCESS && ha->pShunt != NULL)
{
LARGE_INTEGER BytesToCopy;
BytesToCopy.QuadPart = ha->MpqPos.QuadPart - (ha->ShuntPos.QuadPart + sizeof(TMPQShunt));
nError = CopyNonMpqData(ha->hFile, hFile, BytesToCopy);
}
// Write the MPQ header
if(nError == ERROR_SUCCESS)
{
BSWAP_TMPQHEADER(ha->pHeader);
WriteFile(hFile, ha->pHeader, ha->pHeader->dwHeaderSize, &dwTransferred, NULL);
BSWAP_TMPQHEADER(ha->pHeader);
if(dwTransferred != ha->pHeader->dwHeaderSize)
nError = ERROR_DISK_FULL;
}
// Write the data between the header and between the first file
// For this, we have to determine where the first file begins
if(nError == ERROR_SUCCESS)
{
LARGE_INTEGER FirstFilePos;
LARGE_INTEGER TempPos;
TMPQBlockEx * pBlockEx = ha->pExtBlockTable;
TMPQBlock * pBlockEnd = ha->pBlockTable + ha->pHeader->dwBlockTableSize;
TMPQBlock * pBlock = ha->pBlockTable;
// Maximum file position
FirstFilePos.HighPart = 0x7FFFFFFF;
FirstFilePos.LowPart = 0xFFFFFFFF;
// Find the block with the least position in the MPQ
while(pBlock < pBlockEnd)
{
TempPos.HighPart = pBlockEx->wFilePosHigh;
TempPos.LowPart = pBlock->dwFilePos;
if(TempPos.QuadPart < FirstFilePos.QuadPart)
FirstFilePos = TempPos;
pBlockEx++;
pBlock++;
}
// Set the position in the source file right after the file header
TempPos.QuadPart = ha->MpqPos.QuadPart + ha->pHeader->dwHeaderSize;
SetFilePointer(ha->hFile, TempPos.LowPart, &TempPos.HighPart, FILE_BEGIN);
// Get the number of bytes to copy
FirstFilePos.QuadPart -= ha->pHeader->dwHeaderSize;
nError = CopyNonMpqData(ha->hFile, hFile, FirstFilePos);
}
// Now write all file blocks.
if(nError == ERROR_SUCCESS)
nError = CopyMpqFiles(hFile, ha, pFileSeeds);
// Now we need to update the tables positions
// (but only if the tables are at the end of the file)
if(nError == ERROR_SUCCESS)
{
LARGE_INTEGER RelativePos;
LARGE_INTEGER FilePos = {0};
// Set the hash table position
FilePos.LowPart = SetFilePointer(hFile, 0, &FilePos.HighPart, FILE_CURRENT);
RelativePos.QuadPart = FilePos.QuadPart - ha->MpqPos.QuadPart;
ha->pHeader->wHashTablePosHigh = (USHORT)RelativePos.HighPart;
ha->pHeader->dwHashTablePos = RelativePos.LowPart;
ha->HashTablePos = FilePos;
// Set the block table position
RelativePos.QuadPart += ha->pHeader->dwHashTableSize * sizeof(TMPQHash);
FilePos.QuadPart += ha->pHeader->dwHashTableSize * sizeof(TMPQHash);
ha->pHeader->wBlockTablePosHigh = (USHORT)RelativePos.HighPart;
ha->pHeader->dwBlockTablePos = RelativePos.LowPart;
ha->BlockTablePos = FilePos;
// Set the extended block table position
RelativePos.QuadPart += ha->pHeader->dwBlockTableSize * sizeof(TMPQBlock);
FilePos.QuadPart += ha->pHeader->dwBlockTableSize * sizeof(TMPQBlock);
if(ha->ExtBlockTablePos.QuadPart != 0)
{
ha->pHeader->ExtBlockTablePos = RelativePos;
ha->ExtBlockTablePos = FilePos;
RelativePos.QuadPart += ha->pHeader->dwBlockTableSize * sizeof(TMPQBlockEx);
FilePos.QuadPart += ha->pHeader->dwBlockTableSize * sizeof(TMPQBlockEx);
}
// Set the archive size
ha->pHeader->dwArchiveSize = RelativePos.LowPart;
ha->MpqSize = RelativePos;
}
// If succeeded, update the tables in the file
if(nError == ERROR_SUCCESS)
{
CloseHandle(ha->hFile);
ha->FilePointer.QuadPart = 0;
ha->hFile = hFile;
hFile = INVALID_HANDLE_VALUE;
nError = SaveMPQTables(ha);
}
// If all succeeded, switch the archives
if(nError == ERROR_SUCCESS)
{
if(CompactCB != NULL)
CompactCB(lpUserData, CCB_CLOSING_ARCHIVE, 0, 0);
if(!DeleteFile(ha->szFileName) || // Delete the old archive
!CloseHandle(ha->hFile) || // Close the new archive
!MoveFile(szTempFile, ha->szFileName)) // Rename the temporary archive
nError = GetLastError();
}
// Now open the freshly renamed archive file
if(nError == ERROR_SUCCESS)
{
ha->hFile = CreateFile(ha->szFileName, GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
if(ha->hFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
// Invalidate the positions of the archive
if(nError == ERROR_SUCCESS)
{
ha->FilePointer.QuadPart = 0;
ha->pLastFile = NULL;
ha->dwBlockPos = 0;
ha->dwBuffPos = 0;
}
// Cleanup and return
if(hFile != INVALID_HANDLE_VALUE)
CloseHandle(hFile);
if(pFileSeeds != NULL)
FREEMEM(pFileSeeds);
if(nError != ERROR_SUCCESS)
SetLastError(nError);
DeleteFile(szTempFile);
CompactCB = NULL;
return (nError == ERROR_SUCCESS);
}

530
contrib/vmap_extractor_v2/stormlib/SFileCreateArchiveEx.cpp
View file

@ -1,530 +0,0 @@
/*****************************************************************************/
/* SFileCreateArchiveEx.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* MPQ Editing functions */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 24.03.03 1.00 Lad Splitted from SFileOpenArchive.cpp */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
//-----------------------------------------------------------------------------
// Defines
#define DEFAULT_BLOCK_SIZE 3 // Default size of the block
//-----------------------------------------------------------------------------
// Local tables
static DWORD PowersOfTwo[] =
{
0x0000002, 0x0000004, 0x0000008,
0x0000010, 0x0000020, 0x0000040, 0x0000080,
0x0000100, 0x0000200, 0x0000400, 0x0000800,
0x0001000, 0x0002000, 0x0004000, 0x0008000,
0x0010000, 0x0020000, 0x0040000, 0x0080000,
0x0000000
};
/*****************************************************************************/
/* Public functions */
/*****************************************************************************/
//-----------------------------------------------------------------------------
// Opens or creates a (new) MPQ archive.
//
// szMpqName - Name of the archive to be created.
//
// dwCreationDisposition:
//
// Value Archive exists Archive doesn't exist
// ---------- --------------------- ---------------------
// CREATE_NEW Fails Creates new archive
// CREATE_ALWAYS Overwrites existing Creates new archive
// OPEN_EXISTING Opens the archive Fails
// OPEN_ALWAYS Opens the archive Creates new archive
//
// The above mentioned values can be combined with the following flags:
//
// MPQ_CREATE_ARCHIVE_V1 - Creates MPQ archive version 1
// MPQ_CREATE_ARCHIVE_V2 - Creates MPQ archive version 2
//
// dwHashTableSize - Size of the hash table (only if creating a new archive).
// Must be between 2^4 (= 16) and 2^18 (= 262 144)
//
// phMpq - Receives handle to the archive
//
// TODO: Test for archives > 4GB
BOOL WINAPI SFileCreateArchiveEx(const char * szMpqName, DWORD dwCreationDisposition, DWORD dwHashTableSize, HANDLE * phMPQ)
{
LARGE_INTEGER MpqPos = {0}; // Position of MPQ header in the file
TMPQArchive * ha = NULL; // MPQ archive handle
HANDLE hFile = INVALID_HANDLE_VALUE; // File handle
DWORD dwTransferred = 0; // Number of bytes written into the archive
USHORT wFormatVersion;
BOOL bFileExists = FALSE;
int nIndex = 0;
int nError = ERROR_SUCCESS;
// Pre-initialize the result value
if(phMPQ != NULL)
*phMPQ = NULL;
// Check the parameters, if they are valid
if(szMpqName == NULL || *szMpqName == 0 || phMPQ == NULL)
{
SetLastError(ERROR_INVALID_PARAMETER);
return FALSE;
}
// Check the value of dwCreationDisposition against file existence
bFileExists = (GetFileAttributes(szMpqName) != 0xFFFFFFFF);
// Extract format version from the "dwCreationDisposition"
wFormatVersion = (USHORT)(dwCreationDisposition >> 0x10);
dwCreationDisposition &= 0x0000FFFF;
// If the file exists and open required, do it.
if(bFileExists && (dwCreationDisposition == OPEN_EXISTING || dwCreationDisposition == OPEN_ALWAYS))
{
// Try to open the archive normal way. If it fails, it means that
// the file exist, but it is not a MPQ archive.
if(SFileOpenArchiveEx(szMpqName, 0, 0, phMPQ, GENERIC_READ | GENERIC_WRITE))
return TRUE;
}
// Two error cases
if(dwCreationDisposition == CREATE_NEW && bFileExists)
{
SetLastError(ERROR_ALREADY_EXISTS);
return FALSE;
}
if(dwCreationDisposition == OPEN_EXISTING && bFileExists == FALSE)
{
SetLastError(ERROR_FILE_NOT_FOUND);
return FALSE;
}
// At this point, we have to create the archive. If the file exists,
// we will convert it to MPQ archive.
// Check the value of hash table size. It has to be a power of two
// and must be between HASH_TABLE_SIZE_MIN and HASH_TABLE_SIZE_MAX
if(dwHashTableSize < HASH_TABLE_SIZE_MIN)
dwHashTableSize = HASH_TABLE_SIZE_MIN;
if(dwHashTableSize > HASH_TABLE_SIZE_MAX)
dwHashTableSize = HASH_TABLE_SIZE_MAX;
// Round the hash table size up to the nearest power of two
for(nIndex = 0; PowersOfTwo[nIndex] != 0; nIndex++)
{
if(dwHashTableSize <= PowersOfTwo[nIndex])
{
dwHashTableSize = PowersOfTwo[nIndex];
break;
}
}
// Prepare the buffer for decryption engine
if(nError == ERROR_SUCCESS)
nError = PrepareStormBuffer();
// Get the position where the MPQ header will begin.
if(nError == ERROR_SUCCESS)
{
hFile = CreateFile(szMpqName,
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ,
NULL,
dwCreationDisposition,
0,
NULL);
if(hFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
// Retrieve the file size and round it up to 0x200 bytes
if(nError == ERROR_SUCCESS)
{
MpqPos.LowPart = GetFileSize(hFile, (LPDWORD)&MpqPos.HighPart);
MpqPos.QuadPart += 0x1FF;
MpqPos.LowPart &= 0xFFFFFE00;
if(wFormatVersion == MPQ_FORMAT_VERSION_1 && MpqPos.HighPart != 0)
nError = ERROR_DISK_FULL;
if(wFormatVersion == MPQ_FORMAT_VERSION_2 && MpqPos.HighPart > 0x0000FFFF)
nError = ERROR_DISK_FULL;
}
// Move to the end of the file (i.e. begin of the MPQ)
if(nError == ERROR_SUCCESS)
{
if(SetFilePointer(hFile, MpqPos.LowPart, &MpqPos.HighPart, FILE_BEGIN) == 0xFFFFFFFF)
nError = GetLastError();
}
// Set the new end of the file to the MPQ header position
if(nError == ERROR_SUCCESS)
{
if(!SetEndOfFile(hFile))
nError = GetLastError();
}
// Create the archive handle
if(nError == ERROR_SUCCESS)
{
if((ha = ALLOCMEM(TMPQArchive, 1)) == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Fill the MPQ archive handle structure and create the header,
// block buffer, hash table and block table
if(nError == ERROR_SUCCESS)
{
memset(ha, 0, sizeof(TMPQArchive));
strcpy(ha->szFileName, szMpqName);
ha->hFile = hFile;
ha->dwBlockSize = 0x200 << DEFAULT_BLOCK_SIZE;
ha->MpqPos = MpqPos;
ha->FilePointer = MpqPos;
ha->pHeader = &ha->Header;
ha->pHashTable = ALLOCMEM(TMPQHash, dwHashTableSize);
ha->pBlockTable = ALLOCMEM(TMPQBlock, dwHashTableSize);
ha->pExtBlockTable = ALLOCMEM(TMPQBlockEx, dwHashTableSize);
ha->pbBlockBuffer = ALLOCMEM(BYTE, ha->dwBlockSize);
ha->pListFile = NULL;
ha->dwFlags |= MPQ_FLAG_CHANGED;
if(!ha->pHashTable || !ha->pBlockTable || !ha->pExtBlockTable || !ha->pbBlockBuffer)
nError = GetLastError();
hFile = INVALID_HANDLE_VALUE;
}
// Fill the MPQ header and all buffers
if(nError == ERROR_SUCCESS)
{
LARGE_INTEGER TempPos;
TMPQHeader2 * pHeader = ha->pHeader;
DWORD dwHeaderSize = (wFormatVersion == MPQ_FORMAT_VERSION_2) ? sizeof(TMPQHeader2) : sizeof(TMPQHeader);
memset(pHeader, 0, sizeof(TMPQHeader2));
pHeader->dwID = ID_MPQ;
pHeader->dwHeaderSize = dwHeaderSize;
pHeader->dwArchiveSize = pHeader->dwHeaderSize + dwHashTableSize * sizeof(TMPQHash);
pHeader->wFormatVersion = wFormatVersion;
pHeader->wBlockSize = 3; // 0x1000 bytes per block
pHeader->dwHashTableSize = dwHashTableSize;
// Set proper hash table positions
ha->HashTablePos.QuadPart = ha->MpqPos.QuadPart + pHeader->dwHeaderSize;
ha->pHeader->dwHashTablePos = pHeader->dwHeaderSize;
ha->pHeader->wHashTablePosHigh = 0;
// Set proper block table positions
ha->BlockTablePos.QuadPart = ha->HashTablePos.QuadPart +
(ha->pHeader->dwHashTableSize * sizeof(TMPQHash));
TempPos.QuadPart = ha->BlockTablePos.QuadPart - ha->MpqPos.QuadPart;
ha->pHeader->dwBlockTablePos = TempPos.LowPart;
ha->pHeader->wBlockTablePosHigh = (USHORT)TempPos.HighPart;
// For now, we set extended block table positioon top zero unless we add enough
// files to cause the archive size exceed 4 GB
ha->ExtBlockTablePos.QuadPart = 0;
// Clear all tables
memset(ha->pBlockTable, 0, sizeof(TMPQBlock) * dwHashTableSize);
memset(ha->pExtBlockTable, 0, sizeof(TMPQBlockEx) * dwHashTableSize);
memset(ha->pHashTable, 0xFF, sizeof(TMPQHash) * dwHashTableSize);
}
// Write the MPQ header to the file
if(nError == ERROR_SUCCESS)
{
DWORD dwHeaderSize = ha->pHeader->dwHeaderSize;
BSWAP_TMPQHEADER(ha->pHeader);
WriteFile(ha->hFile, ha->pHeader, dwHeaderSize, &dwTransferred, NULL);
BSWAP_TMPQHEADER(ha->pHeader);
if(dwTransferred != ha->pHeader->dwHeaderSize)
nError = ERROR_DISK_FULL;
ha->FilePointer.QuadPart = ha->MpqPos.QuadPart + dwTransferred;
ha->MpqSize.QuadPart += dwTransferred;
}
// Create the internal listfile
if(nError == ERROR_SUCCESS)
nError = SListFileCreateListFile(ha);
// Try to add the internal listfile
if(nError == ERROR_SUCCESS)
SFileAddListFile((HANDLE)ha, NULL);
// Cleanup : If an error, delete all buffers and return
if(nError != ERROR_SUCCESS)
{
FreeMPQArchive(ha);
if(hFile != INVALID_HANDLE_VALUE)
CloseHandle(hFile);
SetLastError(nError);
}
// Return the values
*phMPQ = (HANDLE)ha;
return (nError == ERROR_SUCCESS);
}
//-----------------------------------------------------------------------------
// Changes locale ID of a file
// TODO: Test for archives > 4GB
BOOL WINAPI SFileSetFileLocale(HANDLE hFile, LCID lcNewLocale)
{
TMPQFile * hf = (TMPQFile *)hFile;
// Invalid handle => do nothing
if(IsValidFileHandle(hf) == FALSE || IsValidMpqHandle(hf->ha) == FALSE)
{
SetLastError(ERROR_INVALID_PARAMETER);
return FALSE;
}
// If the file has not been open for writing, do nothing.
if(hf->ha->pListFile == NULL)
return ERROR_ACCESS_DENIED;
hf->pHash->lcLocale = (USHORT)lcNewLocale;
hf->ha->dwFlags |= MPQ_FLAG_CHANGED;
return TRUE;
}
//-----------------------------------------------------------------------------
// Adds a file into the archive
// TODO: Test for archives > 4GB
BOOL WINAPI SFileAddFileEx(HANDLE hMPQ, const char * szFileName, const char * szArchivedName, DWORD dwFlags, DWORD dwQuality, int nFileType)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
HANDLE hFile = INVALID_HANDLE_VALUE;
BOOL bReplaced = FALSE; // TRUE if replacing file in the archive
int nError = ERROR_SUCCESS;
if(nError == ERROR_SUCCESS)
{
// Check valid parameters
if(IsValidMpqHandle(ha) == FALSE || szFileName == NULL || *szFileName == 0 || szArchivedName == NULL || *szArchivedName == 0)
nError = ERROR_INVALID_PARAMETER;
// Check the values of dwFlags
if((dwFlags & MPQ_FILE_COMPRESS_PKWARE) && (dwFlags & MPQ_FILE_COMPRESS_MULTI))
nError = ERROR_INVALID_PARAMETER;
}
// If anyone is trying to add listfile, and the archive already has a listfile,
// deny the operation, but return success.
if(nError == ERROR_SUCCESS)
{
if(ha->pListFile != NULL && !_stricmp(szFileName, LISTFILE_NAME))
return ERROR_SUCCESS;
}
// Open added file
if(nError == ERROR_SUCCESS)
{
hFile = CreateFile(szFileName, GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, 0, NULL);
if(hFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
if(nError == ERROR_SUCCESS)
nError = AddFileToArchive(ha, hFile, szArchivedName, dwFlags, dwQuality, nFileType, &bReplaced);
// Add the file into listfile also
if(nError == ERROR_SUCCESS && bReplaced == FALSE)
nError = SListFileAddNode(ha, szArchivedName);
// Cleanup and exit
if(hFile != INVALID_HANDLE_VALUE)
CloseHandle(hFile);
if(nError != ERROR_SUCCESS)
SetLastError(nError);
return (nError == ERROR_SUCCESS);
}
// Adds a data file into the archive
// TODO: Test for archives > 4GB
BOOL WINAPI SFileAddFile(HANDLE hMPQ, const char * szFileName, const char * szArchivedName, DWORD dwFlags)
{
return SFileAddFileEx(hMPQ, szFileName, szArchivedName, dwFlags, 0, SFILE_TYPE_DATA);
}
// Adds a WAVE file into the archive
// TODO: Test for archives > 4GB
BOOL WINAPI SFileAddWave(HANDLE hMPQ, const char * szFileName, const char * szArchivedName, DWORD dwFlags, DWORD dwQuality)
{
return SFileAddFileEx(hMPQ, szFileName, szArchivedName, dwFlags, dwQuality, SFILE_TYPE_WAVE);
}
//-----------------------------------------------------------------------------
// BOOL SFileRemoveFile(HANDLE hMPQ, char * szFileName)
//
// This function removes a file from the archive. The file content
// remains there, only the entries in the hash table and in the block
// table are updated.
// TODO: Test for archives > 4GB
BOOL WINAPI SFileRemoveFile(HANDLE hMPQ, const char * szFileName, DWORD dwSearchScope)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
TMPQBlockEx * pBlockEx = NULL; // Block entry of deleted file
TMPQBlock * pBlock = NULL; // Block entry of deleted file
TMPQHash * pHash = NULL; // Hash entry of deleted file
DWORD dwBlockIndex = 0;
int nError = ERROR_SUCCESS;
// Check the parameters
if(nError == ERROR_SUCCESS)
{
if(IsValidMpqHandle(ha) == FALSE)
nError = ERROR_INVALID_PARAMETER;
if(dwSearchScope != SFILE_OPEN_BY_INDEX && *szFileName == 0)
nError = ERROR_INVALID_PARAMETER;
}
// Do not allow to remove listfile
if(nError == ERROR_SUCCESS)
{
if(dwSearchScope != SFILE_OPEN_BY_INDEX && !_stricmp(szFileName, LISTFILE_NAME))
nError = ERROR_ACCESS_DENIED;
}
// Get hash entry belonging to this file
if(nError == ERROR_SUCCESS)
{
if((pHash = GetHashEntryEx(ha, (char *)szFileName, lcLocale)) == NULL)
nError = ERROR_FILE_NOT_FOUND;
}
// If index was not found, or is greater than number of files, exit.
if(nError == ERROR_SUCCESS)
{
if((dwBlockIndex = pHash->dwBlockIndex) > ha->pHeader->dwBlockTableSize)
nError = ERROR_FILE_NOT_FOUND;
}
// Get block and test if the file is not already deleted
if(nError == ERROR_SUCCESS)
{
pBlockEx = ha->pExtBlockTable + dwBlockIndex;
pBlock = ha->pBlockTable + dwBlockIndex;
if((pBlock->dwFlags & MPQ_FILE_EXISTS) == 0)
nError = ERROR_FILE_NOT_FOUND;
}
// Now invalidate the block entry and the hash entry. Do not make any
// relocations and file copying, use SFileCompactArchive for it.
if(nError == ERROR_SUCCESS)
{
pBlockEx->wFilePosHigh = 0;
pBlock->dwFilePos = 0;
pBlock->dwFSize = 0;
pBlock->dwCSize = 0;
pBlock->dwFlags = 0;
pHash->dwName1 = 0xFFFFFFFF;
pHash->dwName2 = 0xFFFFFFFF;
pHash->lcLocale = 0xFFFF;
pHash->wPlatform = 0xFFFF;
pHash->dwBlockIndex = HASH_ENTRY_DELETED;
// Update MPQ archive
ha->dwFlags |= MPQ_FLAG_CHANGED;
}
// Remove the file from the list file
if(nError == ERROR_SUCCESS && lcLocale == LANG_NEUTRAL)
nError = SListFileRemoveNode(ha, szFileName);
// Resolve error and exit
if(nError != ERROR_SUCCESS)
SetLastError(nError);
return (nError == ERROR_SUCCESS);
}
// Renames the file within the archive.
// TODO: Test for archives > 4GB
BOOL WINAPI SFileRenameFile(HANDLE hMPQ, const char * szFileName, const char * szNewFileName)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
TMPQHash * pOldHash = NULL; // Hash entry for the original file
TMPQHash * pNewHash = NULL; // Hash entry for the renamed file
DWORD dwBlockIndex = 0;
int nError = ERROR_SUCCESS;
// Test the valid parameters
if(nError == ERROR_SUCCESS)
{
if(hMPQ == NULL || szNewFileName == NULL || *szNewFileName == 0)
nError = ERROR_INVALID_PARAMETER;
if(szFileName == NULL || *szFileName == 0)
nError = ERROR_INVALID_PARAMETER;
}
// Do not allow to rename listfile
if(nError == ERROR_SUCCESS)
{
if(!_stricmp(szFileName, LISTFILE_NAME))
nError = ERROR_ACCESS_DENIED;
}
// Test if the file already exists in the archive
if(nError == ERROR_SUCCESS)
{
if((pNewHash = GetHashEntryEx(ha, szNewFileName, lcLocale)) != NULL)
nError = ERROR_ALREADY_EXISTS;
}
// Get the hash table entry for the original file
if(nError == ERROR_SUCCESS)
{
if((pOldHash = GetHashEntryEx(ha, szFileName, lcLocale)) == NULL)
nError = ERROR_FILE_NOT_FOUND;
}
// Get the hash table entry for the renamed file
if(nError == ERROR_SUCCESS)
{
// Save block table index and remove the hash table entry
dwBlockIndex = pOldHash->dwBlockIndex;
pOldHash->dwName1 = 0xFFFFFFFF;
pOldHash->dwName2 = 0xFFFFFFFF;
pOldHash->lcLocale = 0xFFFF;
pOldHash->wPlatform = 0xFFFF;
pOldHash->dwBlockIndex = HASH_ENTRY_DELETED;
if((pNewHash = FindFreeHashEntry(ha, szNewFileName)) == NULL)
nError = ERROR_CAN_NOT_COMPLETE;
}
// Save the block index and clear the hash entry
if(nError == ERROR_SUCCESS)
{
// Copy the block table index
pNewHash->dwBlockIndex = dwBlockIndex;
ha->dwFlags |= MPQ_FLAG_CHANGED;
}
// Rename the file in the list file
if(nError == ERROR_SUCCESS)
nError = SListFileRenameNode(ha, szFileName, szNewFileName);
// Resolve error and return
if(nError != ERROR_SUCCESS)
SetLastError(nError);
return (nError == ERROR_SUCCESS);
}

63
contrib/vmap_extractor_v2/stormlib/SFileExtractFile.cpp
View file

@ -1,63 +0,0 @@
/*****************************************************************************/
/* SFileExtractFile.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Simple extracting utility */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 20.06.03 1.00 Lad The first version of SFileExtractFile.cpp */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
// TODO: Test for archives > 4GB
BOOL WINAPI SFileExtractFile(HANDLE hMpq, const char * szToExtract, const char * szExtracted)
{
HANDLE hLocalFile = INVALID_HANDLE_VALUE;
HANDLE hMpqFile = NULL;
int nError = ERROR_SUCCESS;
// Create the local file
if(nError == ERROR_SUCCESS)
{
hLocalFile = CreateFile(szExtracted, GENERIC_WRITE, FILE_SHARE_READ, NULL, CREATE_ALWAYS, 0, NULL);
if(hLocalFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
// Open the MPQ file
if(nError == ERROR_SUCCESS)
{
if(!SFileOpenFileEx(hMpq, szToExtract, 0, &hMpqFile))
nError = GetLastError();
}
// Copy the file's content
if(nError == ERROR_SUCCESS)
{
char szBuffer[0x1000];
DWORD dwTransferred = 1;
while(dwTransferred > 0)
{
SFileReadFile(hMpqFile, szBuffer, sizeof(szBuffer), &dwTransferred, NULL);
if(dwTransferred == 0)
break;
WriteFile(hLocalFile, szBuffer, dwTransferred, &dwTransferred, NULL);
if(dwTransferred == 0)
break;
}
}
// Close the files
if(hMpqFile != NULL)
SFileCloseFile(hMpqFile);
if(hLocalFile != INVALID_HANDLE_VALUE)
CloseHandle(hLocalFile);
if(nError != ERROR_SUCCESS)
SetLastError(nError);
return (BOOL)(nError == ERROR_SUCCESS);
}

291
contrib/vmap_extractor_v2/stormlib/SFileFindFile.cpp
View file

@ -1,291 +0,0 @@
/*****************************************************************************/
/* SFileFindFile.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* A module for file searching within MPQs */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 25.03.03 1.00 Lad The first version of SFileFindFile.cpp */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
//-----------------------------------------------------------------------------
// Defines
#define LISTFILE_CACHE_SIZE 0x1000
//-----------------------------------------------------------------------------
// Local functions
static BOOL IsValidSearchHandle(TMPQSearch * hs)
{
if(hs == NULL || IsBadReadPtr(hs, sizeof(TMPQSearch)))
return FALSE;
if(!IsValidMpqHandle(hs->ha))
return FALSE;
return TRUE;
}
// This function compares a string with a wildcard search string.
// returns TRUE, when the string matches with the wildcard.
BOOL CheckWildCard(const char * szString, const char * szWildCard)
{
char * szTemp; // Temporary helper pointer
int nResult = 0; // For memcmp return values
int nMustNotMatch = 0; // Number of following chars int szString,
// which must not match with szWildCard
int nMustMatch = 0; // Number of the following characters,
// which must match
// When the string is empty, it does not match with every wildcard
if(*szString == 0)
return FALSE;
// When the mask is empty, it matches to every wildcard
if(szWildCard == NULL || *szWildCard == 0)
return FALSE;
// Do normal test
for(;;)
{
switch(*szWildCard)
{
case '*': // Means "every number of characters"
// Skip all asterisks
while(*szWildCard == '*')
szWildCard++;
// When no more characters in wildcard, it means that the strings match
if(*szWildCard == 0)
return TRUE;
// The next N characters must not agree
nMustNotMatch |= 0x70000000;
break;
case '?': // Means "One or no character"
while(*szWildCard == '?')
{
nMustNotMatch++;
szWildCard++;
}
break;
default:
// If the two characters match
if(toupper(*szString) == toupper(*szWildCard))
{
// When end of string, they agree
if(*szString == 0)
return TRUE;
nMustNotMatch = 0;
szWildCard++;
szString++;
break;
}
// If the next character must match, the string does not match
if(nMustNotMatch == 0)
return FALSE;
// Count the characters which must match after characters
// that must not match
szTemp = (char *)szWildCard;
nMustMatch = 0;
while(*szTemp != 0 && *szTemp != '*' && *szTemp != '?')
{
nMustMatch++;
szTemp++;
}
// Now skip characters from szString up to number of chars
// that must not match
nResult = -1;
while(nMustNotMatch > 0 && *szString != 0)
{
if((nResult = _strnicmp(szString, szWildCard, nMustMatch)) == 0)
break;
szString++;
nMustNotMatch--;
}
// Make one more comparison
if(nMustNotMatch == 0)
nResult = _strnicmp(szString, szWildCard, nMustMatch);
// If a match has been found, continue the search
if(nResult == 0)
{
nMustNotMatch = 0;
szWildCard += nMustMatch;
szString += nMustMatch;
break;
}
return FALSE;
}
}
}
// Performs one MPQ search
// TODO: Test for archives > 4GB
static int DoMPQSearch(TMPQSearch * hs, SFILE_FIND_DATA * lpFindFileData)
{
TMPQArchive * ha = hs->ha;
TFileNode * pNode;
TMPQHash * pHashEnd = ha->pHashTable + ha->pHeader->dwHashTableSize;
TMPQHash * pHash = ha->pHashTable + hs->dwNextIndex;
// Do until some file found or no more files
while(pHash < pHashEnd)
{
pNode = ha->pListFile[hs->dwNextIndex++];
// If this entry is free, do nothing
if(pHash->dwBlockIndex < HASH_ENTRY_FREE && (DWORD_PTR)pNode < HASH_ENTRY_FREE)
{
// Check the file name.
if(CheckWildCard(pNode->szFileName, hs->szSearchMask))
{
TMPQBlock * pBlock = ha->pBlockTable + pHash->dwBlockIndex;
lpFindFileData->lcLocale = pHash->lcLocale;
lpFindFileData->dwFileSize = pBlock->dwFSize;
lpFindFileData->dwFileFlags = pBlock->dwFlags;
lpFindFileData->dwBlockIndex = pHash->dwBlockIndex;
lpFindFileData->dwCompSize = pBlock->dwCSize;
// Fill the file name and plain file name
strcpy(lpFindFileData->cFileName, pNode->szFileName);
lpFindFileData->szPlainName = strrchr(lpFindFileData->cFileName, '\\');
if(lpFindFileData->szPlainName == NULL)
lpFindFileData->szPlainName = lpFindFileData->cFileName;
else
lpFindFileData->szPlainName++;
// Fill the next entry
return ERROR_SUCCESS;
}
}
pHash++;
}
// No more files found, return error
return ERROR_NO_MORE_FILES;
}
// TODO: Test for archives > 4GB
static void FreeMPQSearch(TMPQSearch *& hs)
{
if(hs != NULL)
{
FREEMEM(hs);
hs = NULL;
}
}
//-----------------------------------------------------------------------------
// Public functions
// TODO: Test for archives > 4GB
HANDLE WINAPI SFileFindFirstFile(HANDLE hMPQ, const char * szMask, SFILE_FIND_DATA * lpFindFileData, const char * szListFile)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
TMPQSearch * hs = NULL; // Search object handle
size_t nSize = 0;
int nError = ERROR_SUCCESS;
// Check for the valid parameters
if(nError == ERROR_SUCCESS)
{
if(!IsValidMpqHandle(ha))
nError = ERROR_INVALID_PARAMETER;
if(szMask == NULL || lpFindFileData == NULL)
nError = ERROR_INVALID_PARAMETER;
if(szListFile == NULL && !IsValidMpqHandle(ha))
nError = ERROR_INVALID_PARAMETER;
}
// Include the listfile into the MPQ's internal listfile
// Note that if the listfile name is NULL, do nothing because the
// internal listfile is always included.
if(nError == ERROR_SUCCESS && szListFile != NULL)
nError = SFileAddListFile((HANDLE)ha, szListFile);
// Allocate the structure for MPQ search
if(nError == ERROR_SUCCESS)
{
nSize = sizeof(TMPQSearch) + strlen(szMask) + 1;
if((hs = (TMPQSearch *)ALLOCMEM(char, nSize)) == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Perform the first search
if(nError == ERROR_SUCCESS)
{
memset(hs, 0, sizeof(TMPQSearch));
hs->ha = ha;
hs->dwNextIndex = 0;
strcpy(hs->szSearchMask, szMask);
nError = DoMPQSearch(hs, lpFindFileData);
}
// Cleanup
if(nError != ERROR_SUCCESS)
{
FreeMPQSearch(hs);
SetLastError(nError);
}
// Return the result value
return (HANDLE)hs;
}
// TODO: Test for archives > 4GB
BOOL WINAPI SFileFindNextFile(HANDLE hFind, SFILE_FIND_DATA * lpFindFileData)
{
TMPQSearch * hs = (TMPQSearch *)hFind;
int nError = ERROR_SUCCESS;
// Check the parameters
if(nError == ERROR_SUCCESS)
{
if(!IsValidSearchHandle(hs) || lpFindFileData == NULL)
nError = ERROR_INVALID_PARAMETER;
}
if(nError == ERROR_SUCCESS)
nError = DoMPQSearch(hs, lpFindFileData);
if(nError != ERROR_SUCCESS)
{
SetLastError(nError);
return FALSE;
}
return TRUE;
}
// TODO: Test for archives > 4GB
BOOL WINAPI SFileFindClose(HANDLE hFind)
{
TMPQSearch * hs = (TMPQSearch *)hFind;
// Check the parameters
if(!IsValidSearchHandle(hs))
{
SetLastError(ERROR_INVALID_PARAMETER);
return FALSE;
}
FreeMPQSearch(hs);
return TRUE;
}

497
contrib/vmap_extractor_v2/stormlib/SFileOpenArchive.cpp
View file

@ -1,497 +0,0 @@
/*****************************************************************************/
/* SFileOpenArchive.cpp Copyright Ladislav Zezula 1999 */
/* */
/* Author : Ladislav Zezula */
/* E-mail : ladik@zezula.net */
/* WWW : www.zezula.net */
/*---------------------------------------------------------------------------*/
/* Archive functions of Storm.dll */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.xx 1.00 Lad The first version of SFileOpenArchive.cpp */
/* 19.11.03 1.01 Dan Big endian handling */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
/*****************************************************************************/
/* Local functions */
/*****************************************************************************/
static BOOL IsAviFile(TMPQHeader * pHeader)
{
DWORD * AviHdr = (DWORD *)pHeader;
// Test for 'RIFF', 'AVI ' or 'LIST'
return (AviHdr[0] == 'FFIR' && AviHdr[2] == ' IVA' && AviHdr[3] == 'TSIL');
}
// This function gets the right positions of the hash table and the block table.
// TODO: Test for archives > 4GB
static int RelocateMpqTablePositions(TMPQArchive * ha)
{
TMPQHeader2 * pHeader = ha->pHeader;
LARGE_INTEGER FileSize;
LARGE_INTEGER TempSize;
// Get the size of the file
FileSize.LowPart = GetFileSize(ha->hFile, (LPDWORD)&FileSize.HighPart);
// Set the proper hash table position
ha->HashTablePos.HighPart = pHeader->wHashTablePosHigh;
ha->HashTablePos.LowPart = pHeader->dwHashTablePos;
ha->HashTablePos.QuadPart += ha->MpqPos.QuadPart;
if(ha->HashTablePos.QuadPart > FileSize.QuadPart)
return ERROR_BAD_FORMAT;
// Set the proper block table position
ha->BlockTablePos.HighPart = pHeader->wBlockTablePosHigh;
ha->BlockTablePos.LowPart = pHeader->dwBlockTablePos;
ha->BlockTablePos.QuadPart += ha->MpqPos.QuadPart;
if(ha->BlockTablePos.QuadPart > FileSize.QuadPart)
return ERROR_BAD_FORMAT;
// Set the proper position of the extended block table
if(pHeader->ExtBlockTablePos.QuadPart != 0)
{
ha->ExtBlockTablePos = pHeader->ExtBlockTablePos;
ha->ExtBlockTablePos.QuadPart += ha->MpqPos.QuadPart;
if(ha->ExtBlockTablePos.QuadPart > FileSize.QuadPart)
return ERROR_BAD_FORMAT;
}
// Size of MPQ archive is computed as the biggest of
// (EndOfBlockTable, EndOfHashTable, EndOfExtBlockTable)
TempSize.QuadPart = ha->HashTablePos.QuadPart + (pHeader->dwHashTableSize * sizeof(TMPQHash));
if(TempSize.QuadPart > ha->MpqSize.QuadPart)
ha->MpqSize = TempSize;
TempSize.QuadPart = ha->BlockTablePos.QuadPart + (pHeader->dwBlockTableSize * sizeof(TMPQBlock));
if(TempSize.QuadPart > ha->MpqSize.QuadPart)
ha->MpqSize = TempSize;
TempSize.QuadPart = ha->ExtBlockTablePos.QuadPart + (pHeader->dwBlockTableSize * sizeof(TMPQBlockEx));
if(TempSize.QuadPart > ha->MpqSize.QuadPart)
ha->MpqSize = TempSize;
// MPQ size does not include the bytes before MPQ header
ha->MpqSize.QuadPart -= ha->MpqPos.QuadPart;
return ERROR_SUCCESS;
}
/*****************************************************************************/
/* Public functions */
/*****************************************************************************/
//-----------------------------------------------------------------------------
// SFileGetLocale and SFileSetLocale
// Set the locale for all neewly opened archives and files
LCID WINAPI SFileGetLocale()
{
return lcLocale;
}
LCID WINAPI SFileSetLocale(LCID lcNewLocale)
{
lcLocale = lcNewLocale;
return lcLocale;
}
//-----------------------------------------------------------------------------
// SFileOpenArchiveEx (not a public function !!!)
//
// szFileName - MPQ archive file name to open
// dwPriority - When SFileOpenFileEx called, this contains the search priority for searched archives
// dwFlags - If contains MPQ_OPEN_NO_LISTFILE, then the internal list file will not be used.
// phMPQ - Pointer to store open archive handle
BOOL SFileOpenArchiveEx(
const char * szMpqName,
DWORD dwPriority,
DWORD dwFlags,
HANDLE * phMPQ,
DWORD dwAccessMode)
{
LARGE_INTEGER TempPos;
TMPQArchive * ha = NULL; // Archive handle
HANDLE hFile = INVALID_HANDLE_VALUE;// Opened archive file handle
DWORD dwMaxBlockIndex = 0; // Maximum value of block entry
DWORD dwBlockTableSize = 0; // Block table size.
DWORD dwTransferred; // Number of bytes read
DWORD dwBytes = 0; // Number of bytes to read
int nError = ERROR_SUCCESS;
// Check the right parameters
if(nError == ERROR_SUCCESS)
{
if(szMpqName == NULL || *szMpqName == 0 || phMPQ == NULL)
nError = ERROR_INVALID_PARAMETER;
}
// Ensure that StormBuffer is allocated
if(nError == ERROR_SUCCESS)
nError = PrepareStormBuffer();
// Open the MPQ archive file
if(nError == ERROR_SUCCESS)
{
hFile = CreateFile(szMpqName, dwAccessMode, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
if(hFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
// Allocate the MPQhandle
if(nError == ERROR_SUCCESS)
{
if((ha = ALLOCMEM(TMPQArchive, 1)) == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Initialize handle structure and allocate structure for MPQ header
if(nError == ERROR_SUCCESS)
{
memset(ha, 0, sizeof(TMPQArchive));
strncpy(ha->szFileName, szMpqName, strlen(szMpqName));
ha->hFile = hFile;
ha->dwPriority = dwPriority;
ha->pHeader = &ha->Header;
ha->pListFile = NULL;
hFile = INVALID_HANDLE_VALUE;
}
// Find the offset of MPQ header within the file
if(nError == ERROR_SUCCESS)
{
LARGE_INTEGER SearchPos = {0};
LARGE_INTEGER MpqPos = {0};
DWORD dwHeaderID;
for(;;)
{
// Invalidate the MPQ ID and read the eventual header
SetFilePointer(ha->hFile, MpqPos.LowPart, &MpqPos.HighPart, FILE_BEGIN);
ReadFile(ha->hFile, ha->pHeader, sizeof(TMPQHeader2), &dwTransferred, NULL);
dwHeaderID = BSWAP_INT32_UNSIGNED(ha->pHeader->dwID);
// Special check : Some MPQs are actually AVI files, only with
// changed extension.
if(MpqPos.QuadPart == 0 && IsAviFile(ha->pHeader))
{
nError = ERROR_AVI_FILE;
break;
}
// If different number of bytes read, break the loop
if(dwTransferred != sizeof(TMPQHeader2))
{
nError = ERROR_BAD_FORMAT;
break;
}
// If there is the MPQ shunt signature, process it
if(dwHeaderID == ID_MPQ_SHUNT && ha->pShunt == NULL)
{
// Fill the shunt header
ha->ShuntPos = MpqPos;
ha->pShunt = &ha->Shunt;
memcpy(ha->pShunt, ha->pHeader, sizeof(TMPQShunt));
BSWAP_TMPQSHUNT(ha->pShunt);
// Set the MPQ pos and repeat the search
MpqPos.QuadPart = SearchPos.QuadPart + ha->pShunt->dwHeaderPos;
continue;
}
// There must be MPQ header signature
if(dwHeaderID == ID_MPQ)
{
BSWAP_TMPQHEADER(ha->pHeader);
// Save the position where the MPQ header has been found
ha->MpqPos = MpqPos;
// If valid signature has been found, break the loop
if(ha->pHeader->wFormatVersion == MPQ_FORMAT_VERSION_1)
{
// W3M Map Protectors set some garbage value into the "dwHeaderSize"
// field of MPQ header. This value is apparently ignored by Storm.dll
if(ha->pHeader->dwHeaderSize != sizeof(TMPQHeader) &&
ha->pHeader->dwHeaderSize != sizeof(TMPQHeader2))
{
ha->dwFlags |= MPQ_FLAG_PROTECTED;
ha->pHeader->dwHeaderSize = sizeof(TMPQHeader);
}
if(ha->pHeader->dwHashTablePos < ha->pHeader->dwArchiveSize &&
ha->pHeader->dwBlockTablePos < ha->pHeader->dwArchiveSize)
{
break;
}
}
if(ha->pHeader->wFormatVersion == MPQ_FORMAT_VERSION_2)
{
break;
}
nError = ERROR_NOT_SUPPORTED;
break;
}
// If a MPQ shunt already has been found,
// and no MPQ header was at potision pointed by the shunt,
// then the archive is corrupt
if(ha->pShunt != NULL)
{
nError = ERROR_BAD_FORMAT;
break;
}
// Move to the next possible offset
SearchPos.QuadPart += 0x200;
MpqPos = SearchPos;
}
}
// Relocate tables position
if(nError == ERROR_SUCCESS)
{
// Clear the fields not supported in older formats
if(ha->pHeader->wFormatVersion < MPQ_FORMAT_VERSION_2)
{
ha->pHeader->ExtBlockTablePos.QuadPart = 0;
ha->pHeader->wBlockTablePosHigh = 0;
ha->pHeader->wHashTablePosHigh = 0;
}
ha->dwBlockSize = (0x200 << ha->pHeader->wBlockSize);
nError = RelocateMpqTablePositions(ha);
}
// Allocate buffers
if(nError == ERROR_SUCCESS)
{
//
// Note that the block table should be as large as the hash table
// (For later file additions).
//
// I have found a MPQ which has the block table larger than
// the hash table. We should avoid buffer overruns caused by that.
//
dwBlockTableSize = max(ha->pHeader->dwHashTableSize, ha->pHeader->dwBlockTableSize);
ha->pHashTable = ALLOCMEM(TMPQHash, ha->pHeader->dwHashTableSize);
ha->pBlockTable = ALLOCMEM(TMPQBlock, dwBlockTableSize);
ha->pExtBlockTable = ALLOCMEM(TMPQBlockEx, dwBlockTableSize);
ha->pbBlockBuffer = ALLOCMEM(BYTE, ha->dwBlockSize);
if(!ha->pHashTable || !ha->pBlockTable || !ha->pExtBlockTable || !ha->pbBlockBuffer)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Read the hash table into memory
if(nError == ERROR_SUCCESS)
{
dwBytes = ha->pHeader->dwHashTableSize * sizeof(TMPQHash);
SetFilePointer(ha->hFile, ha->HashTablePos.LowPart, &ha->HashTablePos.HighPart, FILE_BEGIN);
ReadFile(ha->hFile, ha->pHashTable, dwBytes, &dwTransferred, NULL);
if(dwTransferred != dwBytes)
nError = ERROR_FILE_CORRUPT;
}
// Decrypt hash table and check if it is correctly decrypted
if(nError == ERROR_SUCCESS)
{
TMPQHash * pHashEnd = ha->pHashTable + ha->pHeader->dwHashTableSize;
TMPQHash * pHash;
// We have to convert the hash table from LittleEndian
BSWAP_ARRAY32_UNSIGNED((DWORD *)ha->pHashTable, (dwBytes / sizeof(DWORD)));
DecryptHashTable((DWORD *)ha->pHashTable, (BYTE *)"(hash table)", (ha->pHeader->dwHashTableSize * 4));
// Check hash table if is correctly decrypted
for(pHash = ha->pHashTable; pHash < pHashEnd; pHash++)
{
// Note: Some MPQs from World of Warcraft have wPlatform set to 0x0100.
// If not free or deleted hash entry, check for valid values
if(pHash->dwBlockIndex < HASH_ENTRY_DELETED)
{
// The block index should not be larger than size of the block table
if(pHash->dwBlockIndex > ha->pHeader->dwBlockTableSize)
{
nError = ERROR_BAD_FORMAT;
break;
}
// Remember the highest block table entry
if(pHash->dwBlockIndex > dwMaxBlockIndex)
dwMaxBlockIndex = pHash->dwBlockIndex;
}
}
}
// Now, read the block table
if(nError == ERROR_SUCCESS)
{
memset(ha->pBlockTable, 0, dwBlockTableSize * sizeof(TMPQBlock));
dwBytes = ha->pHeader->dwBlockTableSize * sizeof(TMPQBlock);
SetFilePointer(ha->hFile, ha->BlockTablePos.LowPart, &ha->BlockTablePos.HighPart, FILE_BEGIN);
ReadFile(ha->hFile, ha->pBlockTable, dwBytes, &dwTransferred, NULL);
// We have to convert every DWORD in ha->block from LittleEndian
BSWAP_ARRAY32_UNSIGNED((DWORD *)ha->pBlockTable, dwBytes / sizeof(DWORD));
if(dwTransferred != dwBytes)
nError = ERROR_FILE_CORRUPT;
}
// Decrypt block table.
// Some MPQs don't have Decrypted block table, e.g. cracked Diablo version
// We have to check if block table is really encrypted
if(nError == ERROR_SUCCESS)
{
TMPQBlock * pBlockEnd = ha->pBlockTable + ha->pHeader->dwBlockTableSize;
TMPQBlock * pBlock = ha->pBlockTable;
BOOL bBlockTableEncrypted = FALSE;
// Verify all blocks entries in the table
// The loop usually stops at the first entry
while(pBlock < pBlockEnd)
{
// The lower 8 bits of the MPQ flags are always zero.
// Note that this may change in next MPQ versions
if(pBlock->dwFlags & 0x000000FF)
{
bBlockTableEncrypted = TRUE;
break;
}
// Move to the next block table entry
pBlock++;
}
if(bBlockTableEncrypted)
{
DecryptBlockTable((DWORD *)ha->pBlockTable,
(BYTE *)"(block table)",
(ha->pHeader->dwBlockTableSize * 4));
}
}
// Now, read the extended block table.
// For V1 archives, we still will maintain the extended block table
// (it will be filled with zeros)
// TODO: Test with >4GB
if(nError == ERROR_SUCCESS)
{
memset(ha->pExtBlockTable, 0, dwBlockTableSize * sizeof(TMPQBlockEx));
if(ha->pHeader->ExtBlockTablePos.QuadPart != 0)
{
dwBytes = ha->pHeader->dwBlockTableSize * sizeof(TMPQBlockEx);
SetFilePointer(ha->hFile,
ha->ExtBlockTablePos.LowPart,
&ha->ExtBlockTablePos.HighPart,
FILE_BEGIN);
ReadFile(ha->hFile, ha->pExtBlockTable, dwBytes, &dwTransferred, NULL);
// We have to convert every DWORD in ha->block from LittleEndian
BSWAP_ARRAY16_UNSIGNED((USHORT *)ha->pExtBlockTable, dwBytes / sizeof(USHORT));
// The extended block table is not encrypted (so far)
if(dwTransferred != dwBytes)
nError = ERROR_FILE_CORRUPT;
}
}
// Verify the both block tables (If the MPQ file is not protected)
if(nError == ERROR_SUCCESS && (ha->dwFlags & MPQ_FLAG_PROTECTED) == 0)
{
TMPQBlockEx * pBlockEx = ha->pExtBlockTable;
TMPQBlock * pBlockEnd = ha->pBlockTable + dwMaxBlockIndex + 1;
TMPQBlock * pBlock = ha->pBlockTable;
// If the MPQ file is not protected,
// we will check if all sizes in the block table is correct.
// Note that we will not relocate the block table (change from previous versions)
for(; pBlock < pBlockEnd; pBlock++, pBlockEx++)
{
if(pBlock->dwFlags & MPQ_FILE_EXISTS)
{
// Get the 64-bit file position
TempPos.HighPart = pBlockEx->wFilePosHigh;
TempPos.LowPart = pBlock->dwFilePos;
if(TempPos.QuadPart > ha->MpqSize.QuadPart || pBlock->dwCSize > ha->MpqSize.QuadPart)
{
nError = ERROR_BAD_FORMAT;
break;
}
}
}
}
// If the user didn't specified otherwise,
// include the internal listfile to the TMPQArchive structure
if((dwFlags & MPQ_OPEN_NO_LISTFILE) == 0)
{
if(nError == ERROR_SUCCESS)
SListFileCreateListFile(ha);
// Add the internal listfile
if(nError == ERROR_SUCCESS)
SFileAddListFile((HANDLE)ha, NULL);
}
// Cleanup and exit
if(nError != ERROR_SUCCESS)
{
FreeMPQArchive(ha);
if(hFile != INVALID_HANDLE_VALUE)
CloseHandle(hFile);
SetLastError(nError);
}
else
{
if(pFirstOpen == NULL)
pFirstOpen = ha;
}
*phMPQ = ha;
return (nError == ERROR_SUCCESS);
}
BOOL WINAPI SFileOpenArchive(const char * szMpqName, DWORD dwPriority, DWORD dwFlags, HANDLE * phMPQ)
{
return SFileOpenArchiveEx(szMpqName, dwPriority, dwFlags, phMPQ, GENERIC_READ);
}
//-----------------------------------------------------------------------------
// BOOL SFileCloseArchive(HANDLE hMPQ);
//
// TODO: Test for archives > 4GB
BOOL WINAPI SFileCloseArchive(HANDLE hMPQ)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
if(!IsValidMpqHandle(ha))
{
SetLastError(ERROR_INVALID_PARAMETER);
return FALSE;
}
if(ha->dwFlags & MPQ_FLAG_CHANGED)
{
SListFileSaveToMpq(ha);
SaveMPQTables(ha);
}
FreeMPQArchive(ha);
return TRUE;
}

403
contrib/vmap_extractor_v2/stormlib/SFileOpenFileEx.cpp
View file

@ -1,403 +0,0 @@
/*****************************************************************************/
/* SFileOpenFileEx.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Description : */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.99 1.00 Lad The first version of SFileOpenFileEx.cpp */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
/*****************************************************************************/
/* Local functions */
/*****************************************************************************/
// TODO: Test for archives > 4GB
static BOOL OpenLocalFile(const char * szFileName, HANDLE * phFile)
{
TMPQFile * hf = NULL;
HANDLE hFile = CreateFile(szFileName, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL);
if(hFile != INVALID_HANDLE_VALUE)
{
// Allocate and initialize file handle
size_t nHandleSize = sizeof(TMPQFile) + strlen(szFileName);
if((hf = (TMPQFile *)ALLOCMEM(char, nHandleSize)) != NULL)
{
memset(hf, 0, nHandleSize);
strcpy(hf->szFileName, szFileName);
hf->hFile = hFile;
*phFile = hf;
return TRUE;
}
else
SetLastError(ERROR_NOT_ENOUGH_MEMORY);
}
*phFile = NULL;
return FALSE;
}
// TODO: Test for archives > 4GB
static void FreeMPQFile(TMPQFile *& hf)
{
if(hf != NULL)
{
if(hf->hFile != INVALID_HANDLE_VALUE)
CloseHandle(hf->hFile);
if(hf->pdwBlockPos != NULL)
FREEMEM(hf->pdwBlockPos);
if(hf->pbFileBuffer != NULL)
FREEMEM(hf->pbFileBuffer);
FREEMEM(hf);
hf = NULL;
}
}
/*****************************************************************************/
/* Public functions */
/*****************************************************************************/
//-----------------------------------------------------------------------------
// SFileEnumLocales enums all locale versions within MPQ.
// Functions fills all available language identifiers on a file into the buffer
// pointed by plcLocales. There must be enough entries to copy the localed,
// otherwise the function returns ERROR_INSUFFICIENT_BUFFER.
// TODO: Test for archives > 4GB
int WINAPI SFileEnumLocales(
HANDLE hMPQ,
const char * szFileName,
LCID * plcLocales,
DWORD * pdwMaxLocales,
DWORD dwSearchScope)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
TMPQHash * pHash = NULL;
TMPQHash * pHashEnd = NULL;
DWORD dwLocales = 0;
int nError = ERROR_SUCCESS;
// Test the parameters
if(nError == ERROR_SUCCESS)
{
if(!IsValidMpqHandle(ha) || pdwMaxLocales == NULL)
nError = ERROR_INVALID_PARAMETER;
if(dwSearchScope == SFILE_OPEN_BY_INDEX && (DWORD_PTR)szFileName > ha->pHeader->dwBlockTableSize)
nError = ERROR_INVALID_PARAMETER;
if(dwSearchScope != SFILE_OPEN_BY_INDEX && *szFileName == 0)
nError = ERROR_INVALID_PARAMETER;
}
// Retrieve the hash entry for the required file
if(nError == ERROR_SUCCESS)
{
pHashEnd = ha->pHashTable + ha->pHeader->dwHashTableSize;
if(dwSearchScope == SFILE_OPEN_BY_INDEX)
{
for(pHash = ha->pHashTable; pHash < pHashEnd; pHash++)
{
if(pHash->dwBlockIndex == (DWORD_PTR)szFileName)
break;
}
if(pHash == pHashEnd)
pHash = NULL;
}
else
pHash = GetHashEntry(ha, szFileName);
}
// If the file was not found, sorry
if(nError == ERROR_SUCCESS)
{
if(pHash == NULL)
nError = ERROR_FILE_NOT_FOUND;
}
// Count the entries which correspond to the same file name
if(nError == ERROR_SUCCESS)
{
TMPQHash * pSaveHash = pHash;
DWORD dwName1 = pHash->dwName1;
DWORD dwName2 = pHash->dwName2;
// For nameless access, return 1 locale always
if(dwSearchScope == SFILE_OPEN_BY_INDEX)
dwLocales++;
else
{
while(pHash < pHashEnd && pHash->dwName1 == dwName1 && pHash->dwName2 == dwName2)
{
dwLocales++;
pHash++;
}
}
pHash = pSaveHash;
}
// Test if there is enough space to copy the locales
if(nError == ERROR_SUCCESS)
{
DWORD dwMaxLocales = *pdwMaxLocales;
*pdwMaxLocales = dwLocales;
if(dwMaxLocales < dwLocales)
nError = ERROR_INSUFFICIENT_BUFFER;
}
// Fill all the locales
if(nError == ERROR_SUCCESS)
{
for(DWORD i = 0; i < dwLocales; i++, pHash++)
*plcLocales++ = (LCID)pHash->lcLocale;
}
return nError;
}
//-----------------------------------------------------------------------------
// SFileHasFile
//
// hMPQ - Handle of opened MPQ archive
// szFileName - Name of file to look for
// TODO: Test for archives > 4GB
BOOL WINAPI SFileHasFile(HANDLE hMPQ, char * szFileName)
{
TMPQArchive * ha = (TMPQArchive *)hMPQ;
int nError = ERROR_SUCCESS;
if(nError == ERROR_SUCCESS)
{
if(ha == NULL)
nError = ERROR_INVALID_PARAMETER;
if(*szFileName == 0)
nError = ERROR_INVALID_PARAMETER;
}
// Prepare the file opening
if(nError == ERROR_SUCCESS)
{
if(GetHashEntryEx(ha, szFileName, lcLocale) == NULL)
{
nError = ERROR_FILE_NOT_FOUND;
}
}
// Cleanup
if(nError != ERROR_SUCCESS)
{
SetLastError(nError);
}
return (nError == ERROR_SUCCESS);
}
//-----------------------------------------------------------------------------
// SFileOpenFileEx
//
// hMPQ - Handle of opened MPQ archive
// szFileName - Name of file to open
// dwSearchScope - Where to search
// phFile - Pointer to store opened file handle
// TODO: Test for archives > 4GB
BOOL WINAPI SFileOpenFileEx(HANDLE hMPQ, const char * szFileName, DWORD dwSearchScope, HANDLE * phFile)
{
LARGE_INTEGER FilePos;
TMPQArchive * ha = (TMPQArchive *)hMPQ;
TMPQFile * hf = NULL;
TMPQHash * pHash = NULL; // Hash table index
TMPQBlock * pBlock = NULL; // File block
TMPQBlockEx * pBlockEx = NULL;
DWORD dwHashIndex = 0; // Hash table index
DWORD dwBlockIndex = (DWORD)-1; // Found table index
size_t nHandleSize = 0; // Memory space necessary to allocate TMPQHandle
int nError = ERROR_SUCCESS;
#ifdef _DEBUG
// Due to increasing numbers of files in MPQs, I had to change the behavior
// of opening by file index. Now, the SFILE_OPEN_BY_INDEX value of dwSearchScope
// must be entered. This check will allow to find code places that are incompatible
// with the new behavior.
if(dwSearchScope != SFILE_OPEN_BY_INDEX && szFileName != NULL)
{
assert((DWORD_PTR)szFileName > 0x10000);
}
#endif
if(nError == ERROR_SUCCESS)
{
if(ha == NULL && dwSearchScope == SFILE_OPEN_FROM_MPQ)
nError = ERROR_INVALID_PARAMETER;
if(phFile == NULL)
nError = ERROR_INVALID_PARAMETER;
if(dwSearchScope == SFILE_OPEN_BY_INDEX && (DWORD_PTR)szFileName > ha->pHeader->dwBlockTableSize)
nError = ERROR_INVALID_PARAMETER;
if(dwSearchScope != SFILE_OPEN_BY_INDEX && (szFileName == NULL || *szFileName == 0))
nError = ERROR_INVALID_PARAMETER;
}
// Prepare the file opening
if(nError == ERROR_SUCCESS)
{
// When the file is given by number, ...
if(dwSearchScope == SFILE_OPEN_BY_INDEX)
{
TMPQHash * pHashEnd = ha->pHashTable + ha->pHeader->dwHashTableSize;
// Set handle size to be sizeof(TMPQFile) + length of FileXXXXXXXX.xxx
nHandleSize = sizeof(TMPQFile) + 20;
for(pHash = ha->pHashTable; pHash < pHashEnd; pHash++)
{
if((DWORD_PTR)szFileName == pHash->dwBlockIndex)
{
dwHashIndex = (DWORD)(pHash - ha->pHashTable);
dwBlockIndex = pHash->dwBlockIndex;
break;
}
}
}
else
{
// If we have to open a disk file
if(dwSearchScope == SFILE_OPEN_LOCAL_FILE)
return OpenLocalFile(szFileName, phFile);
nHandleSize = sizeof(TMPQFile) + strlen(szFileName);
if((pHash = GetHashEntryEx(ha, szFileName, lcLocale)) != NULL)
{
dwHashIndex = (DWORD)(pHash - ha->pHashTable);
dwBlockIndex = pHash->dwBlockIndex;
}
}
}
// Get block index from file name and test it
if(nError == ERROR_SUCCESS)
{
// If index was not found, or is greater than number of files, exit.
if(dwBlockIndex == (DWORD)-1 || dwBlockIndex > ha->pHeader->dwBlockTableSize)
nError = ERROR_FILE_NOT_FOUND;
}
// Get block and test if the file was not already deleted.
if(nError == ERROR_SUCCESS)
{
// Get both block tables and file position
pBlockEx = ha->pExtBlockTable + dwBlockIndex;
pBlock = ha->pBlockTable + dwBlockIndex;
FilePos.HighPart = pBlockEx->wFilePosHigh;
FilePos.LowPart = pBlock->dwFilePos;
if(FilePos.QuadPart > ha->MpqSize.QuadPart ||
pBlock->dwCSize > ha->MpqSize.QuadPart)
nError = ERROR_FILE_CORRUPT;
if((pBlock->dwFlags & MPQ_FILE_EXISTS) == 0)
nError = ERROR_FILE_NOT_FOUND;
if(pBlock->dwFlags & ~MPQ_FILE_VALID_FLAGS)
nError = ERROR_NOT_SUPPORTED;
}
// Allocate file handle
if(nError == ERROR_SUCCESS)
{
if((hf = (TMPQFile *)ALLOCMEM(char, nHandleSize)) == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Initialize file handle
if(nError == ERROR_SUCCESS)
{
memset(hf, 0, nHandleSize);
hf->hFile = INVALID_HANDLE_VALUE;
hf->ha = ha;
hf->pBlockEx = pBlockEx;
hf->pBlock = pBlock;
hf->nBlocks = (hf->pBlock->dwFSize + ha->dwBlockSize - 1) / ha->dwBlockSize;
hf->pHash = pHash;
hf->MpqFilePos.HighPart = pBlockEx->wFilePosHigh;
hf->MpqFilePos.LowPart = pBlock->dwFilePos;
hf->MpqFilePos.QuadPart += ha->MpqPos.QuadPart;
hf->dwHashIndex = dwHashIndex;
hf->dwFileIndex = dwBlockIndex;
// Allocate buffers for decompression.
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESSED)
{
// Allocate buffer for block positions. At the begin of file are stored
// DWORDs holding positions of each block relative from begin of file in the archive
// As for newer MPQs, there may be one additional entry in the block table
// (if the MPQ_FILE_HAS_EXTRA flag is set).
// Allocate the buffer to include this DWORD as well
if((hf->pdwBlockPos = ALLOCMEM(DWORD, hf->nBlocks + 2)) == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
// Decrypt file seed. Cannot be used if the file is given by index
if(dwSearchScope != SFILE_OPEN_BY_INDEX)
{
if(hf->pBlock->dwFlags & MPQ_FILE_ENCRYPTED)
{
const char * szTemp = strrchr(szFileName, '\\');
strcpy(hf->szFileName, szFileName);
if(szTemp != NULL)
szFileName = szTemp + 1;
hf->dwSeed1 = DecryptFileSeed((char *)szFileName);
if(hf->pBlock->dwFlags & MPQ_FILE_FIXSEED)
{
hf->dwSeed1 = (hf->dwSeed1 + hf->pBlock->dwFilePos) ^ hf->pBlock->dwFSize;
}
}
}
else
{
// If the file is encrypted and not compressed, we cannot detect the file seed
if(SFileGetFileName(hf, hf->szFileName) == FALSE)
nError = GetLastError();
}
}
// Cleanup
if(nError != ERROR_SUCCESS)
{
FreeMPQFile(hf);
SetLastError(nError);
}
*phFile = hf;
return (nError == ERROR_SUCCESS);
}
//-----------------------------------------------------------------------------
// BOOL SFileCloseFile(HANDLE hFile);
// TODO: Test for archives > 4GB
BOOL WINAPI SFileCloseFile(HANDLE hFile)
{
TMPQFile * hf = (TMPQFile *)hFile;
if(!IsValidFileHandle(hf))
{
SetLastError(ERROR_INVALID_PARAMETER);
return FALSE;
}
// Set the last accessed file in the archive
if(hf->ha != NULL)
hf->ha->pLastFile = NULL;
// Free the structure
FreeMPQFile(hf);
return TRUE;
}

826
contrib/vmap_extractor_v2/stormlib/SFileReadFile.cpp
View file

@ -1,826 +0,0 @@
/*****************************************************************************/
/* SFileReadFile.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Description : */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.99 1.00 Lad The first version of SFileReadFile.cpp */
/* 24.03.99 1.00 Lad Added the SFileGetFileInfo function */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
//-----------------------------------------------------------------------------
// Defines
#define ID_WAVE 0x46464952 // Signature of WAVes for name breaking
#define ID_EXE 0x00005A4D // Signature of executable files
//-----------------------------------------------------------------------------
// Local structures
struct TID2Ext
{
DWORD dwID;
char * szExt;
};
//-----------------------------------------------------------------------------
// ReadMPQBlock
//
// hf - MPQ File handle.
// dwBlockPos - Position of block in the file (relative to file begin)
// buffer - Pointer to target buffer to store blocks.
// dwBlockSize - Number of bytes to read. Must be multiplier of block size.
//
// Returns number of bytes read.
// TODO: Test for archives > 4GB
static DWORD WINAPI ReadMPQBlocks(TMPQFile * hf, DWORD dwBlockPos, BYTE * buffer, DWORD blockBytes)
{
LARGE_INTEGER FilePos;
TMPQArchive * ha = hf->ha; // Archive handle
BYTE * tempBuffer = NULL; // Buffer for reading compressed data from the file
DWORD dwFilePos = dwBlockPos; // Reading position from the file
DWORD dwToRead; // Number of bytes to read
DWORD blockNum; // Block number (needed for decrypt)
DWORD dwBytesRead = 0; // Total number of bytes read
DWORD bytesRemain = 0; // Number of data bytes remaining up to the end of the file
DWORD nBlocks; // Number of blocks to load
DWORD i;
// Test parameters. Block position and block size must be block-aligned, block size nonzero
if((dwBlockPos & (ha->dwBlockSize - 1)) || blockBytes == 0)
return 0;
// Check the end of file
if((dwBlockPos + blockBytes) > hf->pBlock->dwFSize)
blockBytes = hf->pBlock->dwFSize - dwBlockPos;
bytesRemain = hf->pBlock->dwFSize - dwBlockPos;
blockNum = dwBlockPos / ha->dwBlockSize;
nBlocks = blockBytes / ha->dwBlockSize;
if(blockBytes % ha->dwBlockSize)
nBlocks++;
// If file has variable block positions, we have to load them
if((hf->pBlock->dwFlags & MPQ_FILE_COMPRESSED) && hf->bBlockPosLoaded == FALSE)
{
// Move file pointer to the begin of the file in the MPQ
if(hf->MpqFilePos.QuadPart != ha->FilePointer.QuadPart)
{
SetFilePointer(ha->hFile, hf->MpqFilePos.LowPart, &hf->MpqFilePos.HighPart, FILE_BEGIN);
}
// Read block positions from begin of file.
dwToRead = (hf->nBlocks+1) * sizeof(DWORD);
if(hf->pBlock->dwFlags & MPQ_FILE_HAS_EXTRA)
dwToRead += sizeof(DWORD);
// Read the block pos table and convert the buffer to little endian
ReadFile(ha->hFile, hf->pdwBlockPos, dwToRead, &dwBytesRead, NULL);
BSWAP_ARRAY32_UNSIGNED(hf->pdwBlockPos, (hf->nBlocks+1));
//
// If the archive if protected some way, perform additional check
// Sometimes, the file appears not to be encrypted, but it is.
//
// Note: In WoW 1.10+, there's a new flag. With this flag present,
// there's one additional entry in the block table.
//
if(hf->pdwBlockPos[0] != dwBytesRead)
hf->pBlock->dwFlags |= MPQ_FILE_ENCRYPTED;
// Decrypt loaded block positions if necessary
if(hf->pBlock->dwFlags & MPQ_FILE_ENCRYPTED)
{
// If we don't know the file seed, try to find it.
if(hf->dwSeed1 == 0)
hf->dwSeed1 = DetectFileSeed(hf->pdwBlockPos, dwBytesRead);
// If we don't know the file seed, sorry but we cannot extract the file.
if(hf->dwSeed1 == 0)
return 0;
// Decrypt block positions
DecryptMPQBlock(hf->pdwBlockPos, dwBytesRead, hf->dwSeed1 - 1);
// Check if the block positions are correctly decrypted
// I don't know why, but sometimes it will result invalid block positions on some files
if(hf->pdwBlockPos[0] != dwBytesRead)
{
// Try once again to detect file seed and decrypt the blocks
// TODO: Test with >4GB
SetFilePointer(ha->hFile, hf->MpqFilePos.LowPart, &hf->MpqFilePos.HighPart, FILE_BEGIN);
ReadFile(ha->hFile, hf->pdwBlockPos, dwToRead, &dwBytesRead, NULL);
BSWAP_ARRAY32_UNSIGNED(hf->pdwBlockPos, (hf->nBlocks+1));
hf->dwSeed1 = DetectFileSeed(hf->pdwBlockPos, dwBytesRead);
DecryptMPQBlock(hf->pdwBlockPos, dwBytesRead, hf->dwSeed1 - 1);
// Check if the block positions are correctly decrypted
if(hf->pdwBlockPos[0] != dwBytesRead)
return 0;
}
}
// Update hf's variables
ha->FilePointer.QuadPart = hf->MpqFilePos.QuadPart + dwBytesRead;
hf->bBlockPosLoaded = TRUE;
}
// Get file position and number of bytes to read
dwFilePos = dwBlockPos;
dwToRead = blockBytes;
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESSED)
{
dwFilePos = hf->pdwBlockPos[blockNum];
dwToRead = hf->pdwBlockPos[blockNum + nBlocks] - dwFilePos;
}
FilePos.QuadPart = hf->MpqFilePos.QuadPart + dwFilePos;
// Get work buffer for store read data
tempBuffer = buffer;
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESSED)
{
if((tempBuffer = ALLOCMEM(BYTE, dwToRead)) == NULL)
{
SetLastError(ERROR_NOT_ENOUGH_MEMORY);
return 0;
}
}
// Set file pointer, if necessary
if(ha->FilePointer.QuadPart != FilePos.QuadPart)
{
SetFilePointer(ha->hFile, FilePos.LowPart, &FilePos.HighPart, FILE_BEGIN);
}
// 15018F87 : Read all requested blocks
ReadFile(ha->hFile, tempBuffer, dwToRead, &dwBytesRead, NULL);
ha->FilePointer.QuadPart = FilePos.QuadPart + dwBytesRead;
// Block processing part.
DWORD blockStart = 0; // Index of block start in work buffer
DWORD blockSize = min(blockBytes, ha->dwBlockSize);
DWORD index = blockNum; // Current block index
dwBytesRead = 0; // Clear read byte counter
// Walk through all blocks
for(i = 0; i < nBlocks; i++, index++)
{
BYTE * inputBuffer = tempBuffer + blockStart;
int outLength = ha->dwBlockSize;
if(bytesRemain < (DWORD)outLength)
outLength = bytesRemain;
// Get current block length
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESSED)
blockSize = hf->pdwBlockPos[index+1] - hf->pdwBlockPos[index];
// If block is encrypted, we have to decrypt it.
if(hf->pBlock->dwFlags & MPQ_FILE_ENCRYPTED)
{
BSWAP_ARRAY32_UNSIGNED((DWORD *)inputBuffer, blockSize / sizeof(DWORD));
// If we don't know the seed, try to decode it as WAVE file
if(hf->dwSeed1 == 0)
hf->dwSeed1 = DetectFileSeed2((DWORD *)inputBuffer, 3, ID_WAVE, hf->pBlock->dwFSize - 8, 0x45564157);
// Let's try MSVC's standard EXE or header
if(hf->dwSeed1 == 0)
hf->dwSeed1 = DetectFileSeed2((DWORD *)inputBuffer, 2, 0x00905A4D, 0x00000003);
if(hf->dwSeed1 == 0)
return 0;
DecryptMPQBlock((DWORD *)inputBuffer, blockSize, hf->dwSeed1 + index);
BSWAP_ARRAY32_UNSIGNED((DWORD *)inputBuffer, blockSize / sizeof(DWORD));
}
// If the block is really compressed, decompress it.
// WARNING : Some block may not be compressed, it can be determined only
// by comparing uncompressed and compressed size !!!
if(blockSize < (DWORD)outLength)
{
// Is the file compressed with PKWARE Data Compression Library ?
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESS_PKWARE)
Decompress_pklib((char *)buffer, &outLength, (char *)inputBuffer, (int)blockSize);
// Is it a file compressed by Blizzard's multiple compression ?
// Note that Storm.dll v 1.0.9 distributed with Warcraft III
// passes the full path name of the opened archive as the new last parameter
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESS_MULTI)
SCompDecompress((char *)buffer, &outLength, (char *)inputBuffer, (int)blockSize);
dwBytesRead += outLength;
buffer += outLength;
}
else
{
if(buffer != inputBuffer)
memcpy(buffer, inputBuffer, blockSize);
dwBytesRead += blockSize;
buffer += blockSize;
}
blockStart += blockSize;
bytesRemain -= outLength;
}
// Delete input buffer, if necessary
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESSED)
FREEMEM(tempBuffer);
return dwBytesRead;
}
// When this function is called, it is already ensured that the parameters are valid
// (e.g. the "dwToRead + dwFilePos" is not greater than the file size)
// TODO: Test for archives > 4GB
static DWORD WINAPI ReadMPQFileSingleUnit(TMPQFile * hf, DWORD dwFilePos, BYTE * pbBuffer, DWORD dwToRead)
{
TMPQArchive * ha = hf->ha;
DWORD dwBytesRead = 0;
if(ha->FilePointer.QuadPart != hf->MpqFilePos.QuadPart)
{
SetFilePointer(ha->hFile, hf->MpqFilePos.LowPart, &hf->MpqFilePos.HighPart, FILE_BEGIN);
ha->FilePointer = hf->MpqFilePos;
}
// If the file is really compressed, decompress it.
// Otherwise, read the data as-is to the caller.
if(hf->pBlock->dwCSize < hf->pBlock->dwFSize)
{
if(hf->pbFileBuffer == NULL)
{
BYTE * inputBuffer = NULL;
int outputBufferSize = (int)hf->pBlock->dwFSize;
int inputBufferSize = (int)hf->pBlock->dwCSize;
hf->pbFileBuffer = ALLOCMEM(BYTE, outputBufferSize);
inputBuffer = ALLOCMEM(BYTE, inputBufferSize);
if(inputBuffer != NULL && hf->pbFileBuffer != NULL)
{
// Read the compressed file data
ReadFile(ha->hFile, inputBuffer, inputBufferSize, &dwBytesRead, NULL);
// Is the file compressed with PKWARE Data Compression Library ?
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESS_PKWARE)
Decompress_pklib((char *)hf->pbFileBuffer, &outputBufferSize, (char *)inputBuffer, (int)inputBufferSize);
// Is it a file compressed by Blizzard's multiple compression ?
// Note that Storm.dll v 1.0.9 distributed with Warcraft III
// passes the full path name of the opened archive as the new last parameter
if(hf->pBlock->dwFlags & MPQ_FILE_COMPRESS_MULTI)
SCompDecompress((char *)hf->pbFileBuffer, &outputBufferSize, (char *)inputBuffer, (int)inputBufferSize);
}
// Free the temporary buffer
if(inputBuffer != NULL)
FREEMEM(inputBuffer);
}
// Copy the file data, if any there
if(hf->pbFileBuffer != NULL)
{
memcpy(pbBuffer, hf->pbFileBuffer + dwFilePos, dwToRead);
dwBytesRead += dwToRead;
}
}
else
{
// Read the uncompressed file data
ReadFile(ha->hFile, pbBuffer, dwToRead, &dwBytesRead, NULL);
dwBytesRead = (int)dwBytesRead;
}
return (DWORD)dwBytesRead;
}
//-----------------------------------------------------------------------------
// ReadMPQFile
// TODO: Test for archives > 4GB
static DWORD WINAPI ReadMPQFile(TMPQFile * hf, DWORD dwFilePos, BYTE * pbBuffer, DWORD dwToRead)
{
TMPQArchive * ha = hf->ha;
TMPQBlock * pBlock = hf->pBlock; // Pointer to file block
DWORD dwBytesRead = 0; // Number of bytes read from the file
DWORD dwBlockPos; // Position in the file aligned to the whole blocks
DWORD dwLoaded;
// File position is greater or equal to file size ?
if(dwFilePos >= pBlock->dwFSize)
return dwBytesRead;
// If too few bytes in the file remaining, cut them
if((pBlock->dwFSize - dwFilePos) < dwToRead)
dwToRead = (pBlock->dwFSize - dwFilePos);
// If the file is stored as single unit, handle it separately
if(pBlock->dwFlags & MPQ_FILE_SINGLE_UNIT)
return ReadMPQFileSingleUnit(hf, dwFilePos, pbBuffer, dwToRead);
// Block position in the file
dwBlockPos = dwFilePos & ~(ha->dwBlockSize - 1); // Position in the block
// Load the first block, if incomplete. It may be loaded in the cache buffer.
// We have to check if this block is loaded. If not, load it.
if((dwFilePos % ha->dwBlockSize) != 0)
{
// Number of bytes remaining in the buffer
DWORD dwToCopy;
DWORD dwLoaded = ha->dwBlockSize;
// Check if data are loaded in the cache
if(hf != ha->pLastFile || dwBlockPos != ha->dwBlockPos)
{
// Load one MPQ block into archive buffer
dwLoaded = ReadMPQBlocks(hf, dwBlockPos, ha->pbBlockBuffer, ha->dwBlockSize);
if(dwLoaded == 0)
return (DWORD)-1;
// Save lastly accessed file and block position for later use
ha->pLastFile = hf;
ha->dwBlockPos = dwBlockPos;
ha->dwBuffPos = dwFilePos % ha->dwBlockSize;
}
dwToCopy = dwLoaded - ha->dwBuffPos;
if(dwToCopy > dwToRead)
dwToCopy = dwToRead;
// Copy data from block buffer into target buffer
memcpy(pbBuffer, ha->pbBlockBuffer + ha->dwBuffPos, dwToCopy);
// Update pointers
dwToRead -= dwToCopy;
dwBytesRead += dwToCopy;
pbBuffer += dwToCopy;
dwBlockPos += ha->dwBlockSize;
ha->dwBuffPos += dwToCopy;
// If all, return.
if(dwToRead == 0)
return dwBytesRead;
}
// Load the whole ("middle") blocks only if there are more or equal one block
if(dwToRead > ha->dwBlockSize)
{
DWORD dwBlockBytes = dwToRead & ~(ha->dwBlockSize - 1);
dwLoaded = ReadMPQBlocks(hf, dwBlockPos, pbBuffer, dwBlockBytes);
if(dwLoaded == 0)
return (DWORD)-1;
// Update pointers
dwToRead -= dwLoaded;
dwBytesRead += dwLoaded;
pbBuffer += dwLoaded;
dwBlockPos += dwLoaded;
// If all, return.
if(dwToRead == 0)
return dwBytesRead;
}
// Load the terminating block
if(dwToRead > 0)
{
DWORD dwToCopy = ha->dwBlockSize;
// Check if data are loaded in the cache
if(hf != ha->pLastFile || dwBlockPos != ha->dwBlockPos)
{
// Load one MPQ block into archive buffer
dwToCopy = ReadMPQBlocks(hf, dwBlockPos, ha->pbBlockBuffer, ha->dwBlockSize);
if(dwToCopy == 0)
return (DWORD)-1;
// Save lastly accessed file and block position for later use
ha->pLastFile = hf;
ha->dwBlockPos = dwBlockPos;
}
ha->dwBuffPos = 0;
// Check number of bytes read
if(dwToCopy > dwToRead)
dwToCopy = dwToRead;
memcpy(pbBuffer, ha->pbBlockBuffer, dwToCopy);
dwBytesRead += dwToCopy;
ha->dwBuffPos = dwToCopy;
}
// Return what we've read
return dwBytesRead;
}
//-----------------------------------------------------------------------------
// SFileReadFile
// TODO: Test for archives > 4GB
BOOL WINAPI SFileReadFile(HANDLE hFile, VOID * lpBuffer, DWORD dwToRead, DWORD * pdwRead, LPOVERLAPPED lpOverlapped)
{
TMPQFile * hf = (TMPQFile *)hFile;
DWORD dwBytes = 0; // Number of bytes (for everything)
int nError = ERROR_SUCCESS;
// Zero the number of bytes read
if(pdwRead != NULL)
*pdwRead = 0;
// Check valid parameters
if(nError == ERROR_SUCCESS)
{
if(hf == NULL || lpBuffer == NULL)
nError = ERROR_INVALID_PARAMETER;
}
// If direct access to the file, use Win32 for reading
if(nError == ERROR_SUCCESS && hf->hFile != INVALID_HANDLE_VALUE)
{
DWORD dwTransferred;
ReadFile(hf->hFile, lpBuffer, dwToRead, &dwTransferred, lpOverlapped);
if(dwTransferred < dwToRead)
{
SetLastError(ERROR_HANDLE_EOF);
return FALSE;
}
if(pdwRead != NULL)
*pdwRead = dwTransferred;
return TRUE;
}
// Read all the bytes available in the buffer (If any)
if(nError == ERROR_SUCCESS)
{
if(dwToRead > 0)
{
dwBytes = ReadMPQFile(hf, hf->dwFilePos, (BYTE *)lpBuffer, dwToRead);
if(dwBytes == (DWORD)-1)
{
SetLastError(ERROR_CAN_NOT_COMPLETE);
return FALSE;
}
hf->ha->pLastFile = hf;
hf->dwFilePos += dwBytes;
}
if(pdwRead != NULL)
*pdwRead = dwBytes;
}
// Check number of bytes read. If not OK, return FALSE.
if(dwBytes < dwToRead)
{
SetLastError(ERROR_HANDLE_EOF);
return FALSE;
}
return TRUE;
}
//-----------------------------------------------------------------------------
// SFileGetFilePos
//
// Returns position of archive file in the archive (relative to begin of file)
// TODO: Test for archives > 4GB
DWORD WINAPI SFileGetFilePos(HANDLE hFile, DWORD * pdwFilePosHigh)
{
TMPQFile * hf = (TMPQFile *)hFile;
if(pdwFilePosHigh != NULL)
*pdwFilePosHigh = 0;
if(hf == NULL)
{
SetLastError(ERROR_INVALID_PARAMETER);
return (DWORD)-1;
}
// If opened as plain file, ...
if(hf->hFile != INVALID_HANDLE_VALUE)
return 0;
// If opened from archive, return file size
if(pdwFilePosHigh != NULL)
*pdwFilePosHigh = hf->MpqFilePos.HighPart;
return hf->MpqFilePos.LowPart;
}
//-----------------------------------------------------------------------------
// SFileGetFileSize
// TODO: Test for archives > 4GB
DWORD WINAPI SFileGetFileSize(HANDLE hFile, DWORD * pdwFileSizeHigh)
{
TMPQFile * hf = (TMPQFile *)hFile;
if(pdwFileSizeHigh != NULL)
*pdwFileSizeHigh = 0;
if(hf == NULL)
{
SetLastError(ERROR_INVALID_PARAMETER);
return (DWORD)-1;
}
// If opened as plain file, ...
if(hf->hFile != INVALID_HANDLE_VALUE)
return GetFileSize(hf->hFile, pdwFileSizeHigh);
// If opened from archive, return file size
return hf->pBlock->dwFSize;
}
// TODO: Test for archives > 4GB
DWORD WINAPI SFileSetFilePointer(HANDLE hFile, LONG lFilePos, LONG * pdwFilePosHigh, DWORD dwMethod)
{
TMPQArchive * ha;
TMPQFile * hf = (TMPQFile *)hFile;
if(hf == NULL || (pdwFilePosHigh != NULL && *pdwFilePosHigh != 0))
{
SetLastError(ERROR_INVALID_PARAMETER);
return (DWORD)-1;
}
// If opened as plain file, call Win32 API
if(hf->hFile != INVALID_HANDLE_VALUE)
return SetFilePointer(hf->hFile, lFilePos, pdwFilePosHigh, dwMethod);
ha = hf->ha;
switch(dwMethod)
{
case FILE_BEGIN:
// Cannot set pointer before begin of file
if(-lFilePos > (LONG)hf->dwFilePos)
hf->dwFilePos = 0;
else
hf->dwFilePos = lFilePos;
break;
case FILE_CURRENT:
// Cannot set pointer before begin of file
if(-lFilePos > (LONG)hf->dwFilePos)
hf->dwFilePos = 0;
else
hf->dwFilePos += lFilePos;
break;
case FILE_END:
// Cannot set file position before begin of file
if(-lFilePos >= (LONG)hf->pBlock->dwFSize)
hf->dwFilePos = 0;
else
hf->dwFilePos = hf->pBlock->dwFSize + lFilePos;
break;
default:
return ERROR_INVALID_PARAMETER;
}
if(hf == ha->pLastFile && (hf->dwFilePos & ~(ha->dwBlockSize - 1)) == ha->dwBlockPos)
ha->dwBuffPos = hf->dwFilePos & (ha->dwBlockSize - 1);
else
{
ha->pLastFile = NULL;
ha->dwBuffPos = 0;
}
return hf->dwFilePos;
}
//-----------------------------------------------------------------------------
// Tries to retrieve the file name
static TID2Ext id2ext[] =
{
{0x1A51504D, "mpq"}, // MPQ archive header ID ('MPQ\x1A')
{0x46464952, "wav"}, // WAVE header 'RIFF'
{0x324B4D53, "smk"}, // Old "Smacker Video" files 'SMK2'
{0x694B4942, "bik"}, // Bink video files (new)
{0x0801050A, "pcx"}, // PCX images used in Diablo I
{0x544E4F46, "fnt"}, // Font files used in Diablo II
{0x6D74683C, "html"}, // HTML '<htm'
{0x4D54483C, "html"}, // HTML '<HTM
{0x216F6F57, "tbl"}, // Table files
{0x31504C42, "blp"}, // BLP textures
{0x32504C42, "blp"}, // BLP textures (v2)
{0x584C444D, "mdx"}, // MDX files
{0x45505954, "pud"}, // Warcraft II maps
{0x38464947, "gif"}, // GIF images 'GIF8'
{0x3032444D, "m2"}, // WoW ??? .m2
{0x43424457, "dbc"}, // ??? .dbc
{0x47585053, "bls"}, // WoW pixel shaders
{0, NULL} // Terminator
};
// TODO: Test for archives > 4GB
BOOL WINAPI SFileGetFileName(HANDLE hFile, char * szFileName)
{
TMPQFile * hf = (TMPQFile *)hFile; // MPQ File handle
char * szExt = "xxx"; // Default extension
DWORD dwFirstBytes[2]; // The first 4 bytes of the file
DWORD dwFilePos; // Saved file position
int nError = ERROR_SUCCESS;
int i;
// Pre-zero the output buffer
if(szFileName != NULL)
*szFileName = 0;
// Check valid parameters
if(nError == ERROR_SUCCESS)
{
if(hf == NULL || szFileName == NULL)
nError = ERROR_INVALID_PARAMETER;
}
// If the file name is already filled, return it.
if(nError == ERROR_SUCCESS && *hf->szFileName != 0)
{
if(szFileName != hf->szFileName)
strcpy(szFileName, hf->szFileName);
return TRUE;
}
if(nError == ERROR_SUCCESS)
{
if(hf->dwFileIndex == (DWORD)-1)
nError = ERROR_CAN_NOT_COMPLETE;
}
// Read the first 8 bytes from the file
if(nError == ERROR_SUCCESS)
{
dwFirstBytes[0] = dwFirstBytes[1] = 0;
dwFilePos = SFileSetFilePointer(hf, 0, NULL, FILE_CURRENT);
if(!SFileReadFile(hFile, &dwFirstBytes, sizeof(dwFirstBytes), NULL))
nError = GetLastError();
BSWAP_ARRAY32_UNSIGNED(dwFirstBytes, sizeof(dwFirstBytes) / sizeof(DWORD));
SFileSetFilePointer(hf, dwFilePos, NULL, FILE_BEGIN);
}
if(nError == ERROR_SUCCESS)
{
if((dwFirstBytes[0] & 0x0000FFFF) == ID_EXE)
szExt = "exe";
else if(dwFirstBytes[0] == 0x00000006 && dwFirstBytes[1] == 0x00000001)
szExt = "dc6";
else
{
for(i = 0; id2ext[i].szExt != NULL; i++)
{
if(id2ext[i].dwID == dwFirstBytes[0])
{
szExt = id2ext[i].szExt;
break;
}
}
}
// Create the file name
sprintf(hf->szFileName, "File%08lu.%s", hf->dwFileIndex, szExt);
if(szFileName != hf->szFileName)
strcpy(szFileName, hf->szFileName);
}
return (nError == ERROR_SUCCESS);
}
//-----------------------------------------------------------------------------
// Retrieves an information about an archive or about a file within the archive
//
// hMpqOrFile - Handle to an MPQ archive or to a file
// dwInfoType - Information to obtain
// TODO: Test for archives > 4GB
DWORD_PTR WINAPI SFileGetFileInfo(HANDLE hMpqOrFile, DWORD dwInfoType)
{
TMPQArchive * ha = (TMPQArchive *)hMpqOrFile;
TMPQFile * hf = (TMPQFile *)hMpqOrFile;
TMPQBlock * pBlockEnd;
TMPQBlock * pBlock;
DWORD dwFileCount = 0;
DWORD dwSeed;
switch(dwInfoType)
{
case SFILE_INFO_ARCHIVE_SIZE:
if(IsValidMpqHandle(ha))
return ha->pHeader->dwArchiveSize;
break;
case SFILE_INFO_HASH_TABLE_SIZE: // Size of the hash table
if(IsValidMpqHandle(ha))
return ha->pHeader->dwHashTableSize;
break;
case SFILE_INFO_BLOCK_TABLE_SIZE: // Size of the hash table
if(IsValidMpqHandle(ha))
return ha->pHeader->dwBlockTableSize;
break;
case SFILE_INFO_BLOCK_SIZE:
if(IsValidMpqHandle(ha))
return ha->dwBlockSize;
break;
case SFILE_INFO_HASH_TABLE:
if(IsValidMpqHandle(ha))
return (DWORD_PTR)ha->pHashTable;
break;
case SFILE_INFO_BLOCK_TABLE:
if(IsValidMpqHandle(ha))
return (DWORD_PTR)ha->pBlockTable;
break;
case SFILE_INFO_NUM_FILES:
if(IsValidMpqHandle(ha))
{
pBlockEnd = ha->pBlockTable + ha->pHeader->dwBlockTableSize;
for(pBlock = ha->pBlockTable; pBlock < pBlockEnd; pBlock++)
{
if(pBlock->dwFlags & MPQ_FILE_EXISTS)
dwFileCount++;
}
return dwFileCount;
}
break;
case SFILE_INFO_HASH_INDEX:
if(IsValidFileHandle(hf))
return hf->dwHashIndex;
break;
case SFILE_INFO_CODENAME1:
if(IsValidFileHandle(hf))
return hf->pHash->dwName1;
break;
case SFILE_INFO_CODENAME2:
if(IsValidFileHandle(hf))
return hf->pHash->dwName2;
break;
case SFILE_INFO_LOCALEID:
if(IsValidFileHandle(hf))
return hf->pHash->lcLocale;
break;
case SFILE_INFO_BLOCKINDEX:
if(IsValidFileHandle(hf))
return hf->dwFileIndex;
break;
case SFILE_INFO_FILE_SIZE:
if(IsValidFileHandle(hf))
return hf->pBlock->dwFSize;
break;
case SFILE_INFO_COMPRESSED_SIZE:
if(IsValidFileHandle(hf))
return hf->pBlock->dwCSize;
break;
case SFILE_INFO_FLAGS:
if(IsValidFileHandle(hf))
return hf->pBlock->dwFlags;
break;
case SFILE_INFO_POSITION:
if(IsValidFileHandle(hf))
return hf->pBlock->dwFilePos;
break;
case SFILE_INFO_SEED:
if(IsValidFileHandle(hf))
return hf->dwSeed1;
break;
case SFILE_INFO_SEED_UNFIXED:
if(IsValidFileHandle(hf))
{
dwSeed = hf->dwSeed1;
if(hf->pBlock->dwFlags & MPQ_FILE_FIXSEED)
dwSeed = (dwSeed ^ hf->pBlock->dwFSize) - (DWORD)(hf->MpqFilePos.QuadPart - hf->ha->MpqPos.QuadPart);
return dwSeed;
}
break;
}
// Unknown parameter or invalid handle
SetLastError(ERROR_INVALID_PARAMETER);
return 0xFFFFFFFF;
}

561
contrib/vmap_extractor_v2/stormlib/SListFile.cpp
View file

@ -1,561 +0,0 @@
/*****************************************************************************/
/* SListFile.cpp Copyright (c) Ladislav Zezula 2004 */
/*---------------------------------------------------------------------------*/
/* Description: */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 12.06.04 1.00 Lad The first version of SListFile.cpp */
/*****************************************************************************/
#define __STORMLIB_SELF__
#include "StormLib.h"
#include "SCommon.h"
#include <assert.h>
//-----------------------------------------------------------------------------
// Listfile entry structure
#define LISTFILE_CACHE_SIZE 0x1000 // Size of one cache element
#define NO_MORE_CHARACTERS 256
#define HASH_TABLE_SIZE 31 // Initial hash table size (should be a prime number)
// TODO: Check on x64 !!!
#define LISTFILE_ENTRY_DELETED (DWORD_PTR)(-2)
#define LISTFILE_ENTRY_FREE (DWORD_PTR)(-1)
struct TListFileCache
{
HANDLE hFile; // Stormlib file handle
char * szMask; // File mask
DWORD dwFileSize; // Total size of the cached file
DWORD dwBuffSize; // File of the cache
DWORD dwFilePos; // Position of the cache in the file
BYTE * pBegin; // The begin of the listfile cache
BYTE * pPos;
BYTE * pEnd; // The last character in the file cache
BYTE Buffer[1]; // Listfile cache itself
};
//-----------------------------------------------------------------------------
// Local functions (cache)
// Reloads the cache. Returns number of characters
// that has been loaded into the cache.
static int ReloadCache(TListFileCache * pCache)
{
// Check if there is enough characters in the cache
// If not, we have to reload the next block
if(pCache->pPos >= pCache->pEnd)
{
// If the cache is already at the end, do nothing more
if((pCache->dwFilePos + pCache->dwBuffSize) >= pCache->dwFileSize)
return 0;
pCache->dwFilePos += pCache->dwBuffSize;
SFileReadFile(pCache->hFile, pCache->Buffer, pCache->dwBuffSize, &pCache->dwBuffSize, NULL);
if(pCache->dwBuffSize == 0)
return 0;
// Set the buffer pointers
pCache->pBegin =
pCache->pPos = &pCache->Buffer[0];
pCache->pEnd = pCache->pBegin + pCache->dwBuffSize;
}
return pCache->dwBuffSize;
}
static size_t ReadLine(TListFileCache * pCache, char * szLine, int nMaxChars)
{
char * szLineBegin = szLine;
char * szLineEnd = szLine + nMaxChars - 1;
__BeginLoading:
// Skip newlines, spaces, tabs and another non-printable stuff
while(pCache->pPos < pCache->pEnd && *pCache->pPos <= 0x20)
pCache->pPos++;
// Copy the remaining characters
while(pCache->pPos < pCache->pEnd && szLine < szLineEnd)
{
// If we have found a newline, stop loading
if(*pCache->pPos == 0x0D || *pCache->pPos == 0x0A)
break;
*szLine++ = *pCache->pPos++;
}
// If we now need to reload the cache, do it
if(pCache->pPos == pCache->pEnd)
{
if(ReloadCache(pCache) > 0)
goto __BeginLoading;
}
*szLine = 0;
return (szLine - szLineBegin);
}
//-----------------------------------------------------------------------------
// Local functions (listfile nodes)
// This function creates the name for the listfile.
// the file will be created under unique name in the temporary directory
static void GetListFileName(TMPQArchive * /* ha */, char * szListFile)
{
char szTemp[MAX_PATH];
// Create temporary file name int TEMP directory
GetTempPath(sizeof(szTemp)-1, szTemp);
GetTempFileName(szTemp, LISTFILE_NAME, 0, szListFile);
}
// Creates new listfile. The listfile is an array of TListFileNode
// structures. The size of the array is the same like the hash table size,
// the ordering is the same too (listfile item index is the same like
// the index in the MPQ hash table)
int SListFileCreateListFile(TMPQArchive * ha)
{
DWORD dwItems = ha->pHeader->dwHashTableSize;
// The listfile should be NULL now
assert(ha->pListFile == NULL);
ha->pListFile = ALLOCMEM(TFileNode *, dwItems);
if(ha->pListFile == NULL)
return ERROR_NOT_ENOUGH_MEMORY;
memset(ha->pListFile, 0xFF, dwItems * sizeof(TFileNode *));
return ERROR_SUCCESS;
}
// Adds a filename into the listfile. If the file name is already there,
// does nothing.
int SListFileAddNode(TMPQArchive * ha, const char * szFileName)
{
TFileNode * pNode = NULL;
TMPQHash * pHashEnd = ha->pHashTable + ha->pHeader->dwHashTableSize;
TMPQHash * pHash0 = GetHashEntry(ha, szFileName);
TMPQHash * pHash = pHash0;
DWORD dwHashIndex = 0;
size_t nLength; // File name lentgth
DWORD dwName1;
DWORD dwName2;
// If the file does not exist within the MPQ, do nothing
if(pHash == NULL)
return ERROR_SUCCESS;
// If the listfile entry already exists, do nothing
dwHashIndex = (DWORD)(pHash - ha->pHashTable);
dwName1 = pHash->dwName1;
dwName2 = pHash->dwName2;
if((DWORD_PTR)ha->pListFile[dwHashIndex] <= LISTFILE_ENTRY_DELETED)
return ERROR_SUCCESS;
// Create the listfile node and insert it into the listfile table
nLength = strlen(szFileName);
pNode = (TFileNode *)ALLOCMEM(char, sizeof(TFileNode) + nLength);
pNode->dwRefCount = 0;
pNode->nLength = nLength;
strcpy(pNode->szFileName, szFileName);
// Fill the nodes for all language versions
while(pHash->dwBlockIndex < LISTFILE_ENTRY_DELETED)
{
if(pHash->dwName1 == dwName1 && pHash->dwName2 == dwName2)
{
pNode->dwRefCount++;
ha->pListFile[pHash - ha->pHashTable] = pNode;
}
if(++pHash >= pHashEnd)
pHash = ha->pHashTable;
if(pHash == pHash0)
break;
}
return ERROR_SUCCESS;
}
// Removes a filename from the listfile.
// If the name is not there, does nothing
int SListFileRemoveNode(TMPQArchive * ha, const char * szFileName)
{
TFileNode * pNode = NULL;
TMPQHash * pHash = GetHashEntry(ha, szFileName);
size_t nHashIndex = 0;
if(pHash != NULL)
{
nHashIndex = pHash - ha->pHashTable;
pNode = ha->pListFile[nHashIndex];
ha->pListFile[nHashIndex] = (TFileNode *)LISTFILE_ENTRY_DELETED;
// If the reference count has reached zero, do nothing
if(--pNode->dwRefCount == 0)
FREEMEM(pNode);
}
return ERROR_SUCCESS;
}
// Renames a node. We will not deal with the renaming, we'll simply
// remove the old node and insert the new one.
// TODO: Test for archives > 4GB
int SListFileRenameNode(TMPQArchive * ha, const char * szOldFileName, const char * szNewFileName)
{
SListFileRemoveNode(ha, szOldFileName);
return SListFileAddNode(ha, szNewFileName);
}
// TODO: Test for archives > 4GB
int SListFileFreeListFile(TMPQArchive * ha)
{
if(ha->pListFile != NULL)
{
for(DWORD i = 0; i < ha->pHeader->dwHashTableSize; i++)
{
TFileNode * pNode = ha->pListFile[i];
if((DWORD_PTR)pNode < LISTFILE_ENTRY_FREE)
{
if(--pNode->dwRefCount == 0)
{
FREEMEM(pNode);
ha->pListFile[i] = (TFileNode *)LISTFILE_ENTRY_FREE;
}
}
}
FREEMEM(ha->pListFile);
ha->pListFile = NULL;
}
return ERROR_SUCCESS;
}
// Saves the whole listfile into the MPQ.
// TODO: Test for archives > 4GB
int SListFileSaveToMpq(TMPQArchive * ha)
{
TFileNode * pNode = NULL;
TMPQHash * pHashEnd = NULL;
TMPQHash * pHash0 = NULL;
TMPQHash * pHash = NULL;
HANDLE hFile = INVALID_HANDLE_VALUE;
char szListFile[MAX_PATH];
char szBuffer[MAX_PATH+4];
DWORD dwTransferred;
size_t nLength = 0;
DWORD dwName1 = 0;
DWORD dwName2 = 0;
LCID lcSave = lcLocale;
int nError = ERROR_SUCCESS;
// If no listfile, do nothing
if(ha->pListFile == NULL)
return ERROR_SUCCESS;
// Create the local listfile
if(nError == ERROR_SUCCESS)
{
GetListFileName(ha, szListFile);
hFile = CreateFile(szListFile, GENERIC_READ | GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_FLAG_DELETE_ON_CLOSE, NULL);
if(hFile == INVALID_HANDLE_VALUE)
nError = GetLastError();
}
// Find the hash entry corresponding to listfile
pHashEnd = ha->pHashTable + ha->pHeader->dwHashTableSize;
pHash0 = pHash = GetHashEntry(ha, 0);
if(pHash == NULL)
pHash0 = pHash = ha->pHashTable;
// Save the file
if(nError == ERROR_SUCCESS)
{
for(;;)
{
if(pHash->dwName1 != dwName1 && pHash->dwName2 != dwName2 && pHash->dwBlockIndex < LISTFILE_ENTRY_DELETED)
{
dwName1 = pHash->dwName1;
dwName2 = pHash->dwName2;
pNode = ha->pListFile[pHash - ha->pHashTable];
if((DWORD_PTR)pNode < LISTFILE_ENTRY_DELETED)
{
memcpy(szBuffer, pNode->szFileName, pNode->nLength);
szBuffer[pNode->nLength + 0] = 0x0D;
szBuffer[pNode->nLength + 1] = 0x0A;
WriteFile(hFile, szBuffer, (DWORD)(pNode->nLength + 2), &dwTransferred, NULL);
}
}
if(++pHash >= pHashEnd)
pHash = ha->pHashTable;
if(pHash == pHash0)
break;
}
// Write the listfile name (if not already there)
if(GetHashEntry(ha, LISTFILE_NAME) == NULL)
{
nLength = strlen(LISTFILE_NAME);
memcpy(szBuffer, LISTFILE_NAME, nLength);
szBuffer[nLength + 0] = 0x0D;
szBuffer[nLength + 1] = 0x0A;
WriteFile(hFile, szBuffer, (DWORD)(nLength + 2), &dwTransferred, NULL);
}
// Add the listfile into the archive.
SFileSetLocale(LANG_NEUTRAL);
nError = AddFileToArchive(ha, hFile, LISTFILE_NAME, MPQ_FILE_COMPRESS_PKWARE | MPQ_FILE_ENCRYPTED | MPQ_FILE_REPLACEEXISTING, 0, SFILE_TYPE_DATA, NULL);
}
// Close the temporary file. This will delete it too.
if(hFile != INVALID_HANDLE_VALUE)
CloseHandle(hFile);
lcLocale = lcSave;
return nError;
}
//-----------------------------------------------------------------------------
// File functions
// Adds a listfile into the MPQ archive.
// Note that the function does not remove the
// TODO: Test for archives > 4GB
int WINAPI SFileAddListFile(HANDLE hMpq, const char * szListFile)
{
TListFileCache * pCache = NULL;
TMPQArchive * ha = (TMPQArchive *)hMpq;
HANDLE hListFile = NULL;
char szFileName[MAX_PATH + 1];
DWORD dwSearchScope = SFILE_OPEN_LOCAL_FILE;
DWORD dwCacheSize = 0;
DWORD dwFileSize = 0;
size_t nLength = 0;
int nError = ERROR_SUCCESS;
// If the szListFile is NULL, it means we have to open internal listfile
if(szListFile == NULL)
{
szListFile = LISTFILE_NAME;
dwSearchScope = SFILE_OPEN_FROM_MPQ;
}
// Open the local/internal listfile
if(nError == ERROR_SUCCESS)
{
if(!SFileOpenFileEx((HANDLE)ha, szListFile, dwSearchScope, &hListFile))
nError = GetLastError();
}
if(nError == ERROR_SUCCESS)
{
dwCacheSize =
dwFileSize = SFileGetFileSize(hListFile, NULL);
// Try to allocate memory for the complete file. If it fails,
// load the part of the file
pCache = (TListFileCache *)ALLOCMEM(char, (sizeof(TListFileCache) + dwCacheSize));
if(pCache == NULL)
{
dwCacheSize = LISTFILE_CACHE_SIZE;
pCache = (TListFileCache *)ALLOCMEM(char, sizeof(TListFileCache) + dwCacheSize);
}
if(pCache == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
if(nError == ERROR_SUCCESS)
{
// Initialize the file cache
memset(pCache, 0, sizeof(TListFileCache));
pCache->hFile = hListFile;
pCache->dwFileSize = dwFileSize;
pCache->dwBuffSize = dwCacheSize;
pCache->dwFilePos = 0;
// Fill the cache
SFileReadFile(hListFile, pCache->Buffer, pCache->dwBuffSize, &pCache->dwBuffSize, NULL);
// Initialize the pointers
pCache->pBegin =
pCache->pPos = &pCache->Buffer[0];
pCache->pEnd = pCache->pBegin + pCache->dwBuffSize;
// Load the node tree
while((nLength = ReadLine(pCache, szFileName, sizeof(szFileName) - 1)) > 0)
SListFileAddNode(ha, szFileName);
// Add well-known names
// Sometimes, they are not in listfile, but they exist in the archive
SListFileAddNode(ha, LISTFILE_NAME);
SListFileAddNode(ha, SIGNATURE_NAME);
SListFileAddNode(ha, ATTRIBUTES_NAME);
}
// Cleanup & exit
if(pCache != NULL)
SListFileFindClose((HANDLE)pCache);
return nError;
}
//-----------------------------------------------------------------------------
// Passing through the listfile
// TODO: Test for archives > 4GB
HANDLE SListFileFindFirstFile(HANDLE hMpq, const char * szListFile, const char * szMask, SFILE_FIND_DATA * lpFindFileData)
{
TListFileCache * pCache = NULL;
TMPQArchive * ha = (TMPQArchive *)hMpq;
HANDLE hListFile = NULL;
DWORD dwSearchScope = SFILE_OPEN_LOCAL_FILE;
DWORD dwCacheSize = 0;
DWORD dwFileSize = 0;
size_t nLength = 0;
int nError = ERROR_SUCCESS;
// Initialize the structure with zeros
memset(lpFindFileData, 0, sizeof(SFILE_FIND_DATA));
// If the szListFile is NULL, it means we have to open internal listfile
if(szListFile == NULL)
{
szListFile = LISTFILE_NAME;
dwSearchScope = SFILE_OPEN_FROM_MPQ;
}
// Open the local/internal listfile
if(nError == ERROR_SUCCESS)
{
if(!SFileOpenFileEx((HANDLE)ha, szListFile, dwSearchScope, &hListFile))
nError = GetLastError();
}
if(nError == ERROR_SUCCESS)
{
dwCacheSize =
dwFileSize = SFileGetFileSize(hListFile, NULL);
// Try to allocate memory for the complete file. If it fails,
// load the part of the file
pCache = (TListFileCache *)ALLOCMEM(char, sizeof(TListFileCache) + dwCacheSize);
if(pCache == NULL)
{
dwCacheSize = LISTFILE_CACHE_SIZE;
pCache = (TListFileCache *)ALLOCMEM(char, sizeof(TListFileCache) + dwCacheSize);
}
if(pCache == NULL)
nError = ERROR_NOT_ENOUGH_MEMORY;
}
if(nError == ERROR_SUCCESS)
{
// Initialize the file cache
memset(pCache, 0, sizeof(TListFileCache));
pCache->hFile = hListFile;
pCache->dwFileSize = dwFileSize;
pCache->dwBuffSize = dwCacheSize;
pCache->dwFilePos = 0;
if(szMask != NULL)
{
pCache->szMask = ALLOCMEM(char, strlen(szMask) + 1);
strcpy(pCache->szMask, szMask);
}
// Fill the cache
SFileReadFile(hListFile, pCache->Buffer, pCache->dwBuffSize, &pCache->dwBuffSize, NULL);
// Initialize the pointers
pCache->pBegin =
pCache->pPos = &pCache->Buffer[0];
pCache->pEnd = pCache->pBegin + pCache->dwBuffSize;
for(;;)
{
// Read the (next) line
nLength = ReadLine(pCache, lpFindFileData->cFileName, sizeof(lpFindFileData->cFileName));
if(nLength == 0)
{
nError = ERROR_NO_MORE_FILES;
break;
}
// If some mask entered, check it
if(CheckWildCard(lpFindFileData->cFileName, pCache->szMask))
break;
}
}
// Cleanup & exit
if(nError != ERROR_SUCCESS)
{
memset(lpFindFileData, 0, sizeof(SFILE_FIND_DATA));
SListFileFindClose((HANDLE)pCache);
pCache = NULL;
SetLastError(nError);
}
return (HANDLE)pCache;
}
// TODO: Test for archives > 4GB
BOOL SListFileFindNextFile(HANDLE hFind, SFILE_FIND_DATA * lpFindFileData)
{
TListFileCache * pCache = (TListFileCache *)hFind;
size_t nLength;
BOOL bResult = FALSE;
int nError = ERROR_SUCCESS;
for(;;)
{
// Read the (next) line
nLength = ReadLine(pCache, lpFindFileData->cFileName, sizeof(lpFindFileData->cFileName));
if(nLength == 0)
{
nError = ERROR_NO_MORE_FILES;
break;
}
// If some mask entered, check it
if(CheckWildCard(lpFindFileData->cFileName, pCache->szMask))
{
bResult = TRUE;
break;
}
}
if(nError != ERROR_SUCCESS)
SetLastError(nError);
return bResult;
}
// TODO: Test for archives > 4GB
BOOL SListFileFindClose(HANDLE hFind)
{
TListFileCache * pCache = (TListFileCache *)hFind;
if(pCache != NULL)
{
if(pCache->hFile != NULL)
SFileCloseFile(pCache->hFile);
if(pCache->szMask != NULL)
FREEMEM(pCache->szMask);
FREEMEM(pCache);
return TRUE;
}
return FALSE;
}

67
contrib/vmap_extractor_v2/stormlib/StormDll.h
View file

@ -1,67 +0,0 @@
/*****************************************************************************/
/* Storm.h Copyright Justin Olbrantz(Quantam) 2000 */
/*---------------------------------------------------------------------------*/
/* Storm Interface Library v1.0 for Windows */
/* */
/* Author : Justin Olbrantz(Quantam) */
/* E-mail : omega@dragonfire.net */
/* WWW : www.campaigncreations.com/starcraft/mpq2k/inside_mopaq/ */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.00 1.00 Qua The first version of Storm.h */
/* 11.04.03 1.00 Lad Added some functions */
/*****************************************************************************/
// We need the Windows data types for the Storm prototypes
#include <windows.h>
#ifndef __STORM_H__
#define __STORM_H__
// Somethimes is necessary to change the function names so they
// will not conflict with other MPQ tools.
#ifdef STORM_ALTERNATE_NAMES
#define SFILE(Name) Storm##Name
#define SCOMP(Name) Storm##Name
#else
#define SFILE(Name) SFile##Name
#define SCOMP(Name) SComp##Name
#endif
// Just in case anyone is still using C out there
#ifdef __cplusplus
extern "C" {
#endif
// Storm file function prototypes
BOOL WINAPI SFILE(OpenArchive)(LPCSTR lpFileName, DWORD dwPriority, DWORD dwFlags, HANDLE *hMPQ);
BOOL WINAPI SFILE(CloseArchive)(HANDLE hMPQ);
BOOL WINAPI SFILE(GetArchiveName)(HANDLE hMPQ, LPCSTR lpBuffer, DWORD dwBufferLength);
BOOL WINAPI SFILE(OpenFile)(LPCSTR lpFileName, HANDLE *hFile);
BOOL WINAPI SFILE(OpenFileEx)(HANDLE hMPQ, LPCSTR lpFileName, DWORD dwSearchScope, HANDLE *hFile);
BOOL WINAPI SFILE(CloseFile)(HANDLE hFile);
DWORD WINAPI SFILE(GetFileSize)(HANDLE hFile, LPDWORD lpFileSizeHigh);
BOOL WINAPI SFILE(GetFileArchive)(HANDLE hFile, HANDLE *hMPQ);
BOOL WINAPI SFILE(GetFileName)(HANDLE hFile, LPCSTR lpBuffer, DWORD dwBufferLength);
DWORD WINAPI SFILE(SetFilePointer)(HANDLE hFile, long lDistanceToMove, PLONG lplDistanceToMoveHigh, DWORD dwMoveMethod);
BOOL WINAPI SFILE(ReadFile)(HANDLE hFile,LPVOID lpBuffer,DWORD nNumberOfBytesToRead,LPDWORD lpNumberOfBytesRead,LPOVERLAPPED lpOverlapped);
LCID WINAPI SFILE(SetLocale)(LCID nNewLocale);
BOOL WINAPI SFILE(GetBasePath)(LPCSTR lpBuffer, DWORD dwBufferLength);
BOOL WINAPI SFILE(SetBasePath)(LPCSTR lpNewBasePath);
// Storm (de)compression functions
BOOL WINAPI SCOMP(Compress) (char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int uCmp, int uCmpType, int nCmpLevel);
BOOL WINAPI SCOMP(Decompress)(char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength);
#if defined(_MSC_VER) && !defined(BUILDING_STORM_CPP)
#pragma comment(lib, "Storm.lib") // Force linking Storm.lib and thus Storm.dll
#endif
#ifdef __cplusplus
}
#endif
#endif // __STORM_H__

579
contrib/vmap_extractor_v2/stormlib/StormLib.h
View file

@ -1,579 +0,0 @@
/*****************************************************************************/
/* StormLib.h Copyright (c) Ladislav Zezula 1999-2005 */
/*---------------------------------------------------------------------------*/
/* StormLib library v 5.00 */
/* */
/* Author : Ladislav Zezula */
/* E-mail : ladik@zezula.net */
/* WWW : http://www.zezula.net */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.99 1.00 Lad Created */
/* 24.03.03 2.50 Lad Version 2.50 */
/* 02.04.03 3.00 Lad Version 3.00 with compression */
/* 11.04.03 3.01 Lad Renamed to StormLib.h for compatibility with */
/* original headers for Storm.dll */
/* 10.05.03 3.02 Lad Added Pkware DCL compression */
/* 26.05.03 4.00 Lad Completed all compressions */
/* 18.06.03 4.01 Lad Added SFileSetFileLocale */
/* Added SFileExtractFile */
/* 26.07.03 4.02 Lad Implemented nameless rename and delete */
/* 26.07.03 4.03 Lad Added support for protected MPQs */
/* 28.08.03 4.10 Lad Fixed bugs that caused StormLib incorrectly work */
/* with Diablo I savegames and with files having full */
/* hash table */
/* 08.12.03 4.11 DCH Fixed bug in reading file block larger than 0x1000 */
/* on certain files. */
/* Fixed bug in AddFile with MPQ_FILE_REPLACE_EXISTING */
/* (Thanx Daniel Chiamarello, dchiamarello@madvawes.com)*/
/* 21.12.03 4.50 Lad Completed port for Mac */
/* Fixed bug in compacting (if fsize is mul of 0x1000) */
/* Fixed bug in SCompCompress */
/* 27.05.04 4.51 Lad Changed memory management from new/delete to our */
/* own macros */
/* 22.06.04 4.60 Lad Optimized search. Support for multiple listfiles. */
/* 30.09.04 4.61 Lad Fixed some bugs (Aaargh !!!) */
/* Correctly works if HashTableSize > BlockTableSize */
/* 29.12.04 4.70 Lad Fixed compatibility problem with MPQs from WoW */
/* 14.07.05 5.00 Lad Added the BZLIB compression support */
/* Added suport of files stored as single unit */
/* 17.04.06 5.01 Lad Converted to MS Visual Studio 8.0 */
/* Fixed issue with protected Warcraft 3 protected maps */
/* 15.05.06 5.02 Lad Fixed issue with WoW 1.10+ */
/* 07.09.06 5.10 Lad Fixed processing files longer than 2GB */
/* 22.11.06 6.00 Lad Support for MPQ archives V2 */
/*****************************************************************************/
#ifndef __STORMLIB_H_
#define __STORMLIB_H_
#include "StormPort.h"
//-----------------------------------------------------------------------------
// Use the apropriate library
//
// The library type is encoded in the library name as the following
// StormLibXYZ.lib
//
// X - D for Debug version, R for Release version
// Y - A for ANSI version, U for Unicode version (Unicode version does not exist yet)
// Z - S for static C library, D for multithreaded DLL C-library
//
#if defined(_MSC_VER) && !defined (__STORMLIB_SELF__)
#ifdef _DEBUG // DEBUG VERSIONS
#ifdef _DLL
#pragma comment(lib, "StormLibDAD.lib") // Debug Ansi Dynamic version
#else
#pragma comment(lib, "StormLibDAS.lib") // Debug Ansi Static version
#endif
#else // RELEASE VERSIONS
#ifdef _DLL
#pragma comment(lib, "StormLibRAD.lib") // Release Ansi Dynamic version
#else
#pragma comment(lib, "StormLibRAS.lib") // Release Ansi Static version
#endif
#endif
#endif
//-----------------------------------------------------------------------------
// Defines
#define ID_MPQ 0x1A51504D // MPQ archive header ID ('MPQ\x1A')
#define ID_MPQ_SHUNT 0x1B51504D // MPQ shunt entry ('MPQ\x1B')
#define ERROR_AVI_FILE 10000 // No MPQ file, but AVI file.
// Values for SFileCreateArchiveEx
#define HASH_TABLE_SIZE_MIN 0x00002
#define HASH_TABLE_SIZE_MAX 0x40000
#define HASH_ENTRY_DELETED 0xFFFFFFFE // Block index for deleted hash entry
#define HASH_ENTRY_FREE 0xFFFFFFFF // Block index for free hash entry
// Values for SFileOpenArchive
#define SFILE_OPEN_HARD_DISK_FILE 2 // Open the archive on HDD
#define SFILE_OPEN_CDROM_FILE 3 // Open the archive only if it is on CDROM
// Values for SFileOpenFile
#define SFILE_OPEN_FROM_MPQ 0 // Open the file from the MPQ archive
#define SFILE_OPEN_BY_INDEX 1 // The 'szFileName' parameter is actually the file index
#define SFILE_OPEN_LOCAL_FILE (DWORD)-1 // Open the file from the MPQ archive
// Flags for TMPQArchive::dwFlags
#define MPQ_FLAG_CHANGED 0x00000001 // If set, the MPQ has been changed
#define MPQ_FLAG_PROTECTED 0x00000002 // Set on protected MPQs (like W3M maps)
// Flags for SFileAddFile
#define MPQ_FILE_COMPRESS_PKWARE 0x00000100 // Compression made by PKWARE Data Compression Library
#define MPQ_FILE_COMPRESS_MULTI 0x00000200 // Multiple compressions
#define MPQ_FILE_COMPRESSED 0x0000FF00 // File is compressed
#define MPQ_FILE_ENCRYPTED 0x00010000 // Indicates whether file is encrypted
#define MPQ_FILE_FIXSEED 0x00020000 // File decrypt seed has to be fixed
#define MPQ_FILE_SINGLE_UNIT 0x01000000 // File is stored as a single unit, rather than split into sectors (Thx, Quantam)
#define MPQ_FILE_DUMMY_FILE 0x02000000 // The file is only 1 byte long and its name is a hash
#define MPQ_FILE_HAS_EXTRA 0x04000000 // The file has extra data appended after regular data.
// Must be with compressed files only
#define MPQ_FILE_EXISTS 0x80000000 // Set if file exists, reset when the file was deleted
#define MPQ_FILE_REPLACEEXISTING 0x80000000 // Replace when the file exist (SFileAddFile)
#define MPQ_FILE_VALID_FLAGS (MPQ_FILE_COMPRESS_PKWARE | \
MPQ_FILE_COMPRESS_MULTI | \
MPQ_FILE_ENCRYPTED | \
MPQ_FILE_FIXSEED | \
MPQ_FILE_SINGLE_UNIT | \
MPQ_FILE_DUMMY_FILE | \
MPQ_FILE_HAS_EXTRA | \
MPQ_FILE_EXISTS)
// Compression types for multilpe compressions
#define MPQ_COMPRESSION_HUFFMANN 0x01 // Huffmann compression (used on WAVE files only)
#define MPQ_COMPRESSION_ZLIB 0x02 // ZLIB compression
#define MPQ_COMPRESSION_PKWARE 0x08 // PKWARE DCL compression
#define MPQ_COMPRESSION_BZIP2 0x10 // BZIP2 compression
#define MPQ_COMPRESSION_WAVE_MONO 0x40 //
#define MPQ_COMPRESSION_WAVE_STEREO 0x80 //
// Constants for SFileAddWave
#define MPQ_WAVE_QUALITY_HIGH 0 // Best quality, the worst compression
#define MPQ_WAVE_QUALITY_MEDIUM 1 // Medium quality, medium compression
#define MPQ_WAVE_QUALITY_LOW 2 // Low quality, the best compression
// Constants for SFileGetFileInfo
#define SFILE_INFO_ARCHIVE_SIZE 1 // MPQ size (value from header)
#define SFILE_INFO_HASH_TABLE_SIZE 2 // Size of hash table, in entries
#define SFILE_INFO_BLOCK_TABLE_SIZE 3 // Number of entries in the block table
#define SFILE_INFO_BLOCK_SIZE 4 // Size of file block (in bytes)
#define SFILE_INFO_HASH_TABLE 5 // Pointer to Hash table (TMPQHash *)
#define SFILE_INFO_BLOCK_TABLE 6 // Pointer to Block Table (TMPQBlock *)
#define SFILE_INFO_NUM_FILES 7 // Real number of files within archive
//------
#define SFILE_INFO_HASH_INDEX 8 // Hash index of file in MPQ
#define SFILE_INFO_CODENAME1 9 // The first codename of the file
#define SFILE_INFO_CODENAME2 10 // The second codename of the file
#define SFILE_INFO_LOCALEID 11 // Locale ID of file in MPQ
#define SFILE_INFO_BLOCKINDEX 12 // Index to Block Table
#define SFILE_INFO_FILE_SIZE 13 // Original file size
#define SFILE_INFO_COMPRESSED_SIZE 14 // Compressed file size
#define SFILE_INFO_FLAGS 15 // File flags
#define SFILE_INFO_POSITION 16 // File position within archive
#define SFILE_INFO_SEED 17 // File decryption seed
#define SFILE_INFO_SEED_UNFIXED 18 // Decryption seed not fixed to file pos and size
// Values for compact callback
#define CCB_CHECKING_FILES 1 // Checking archive (dwParam1 = current, dwParam2 = total)
#define CCB_CHECKING_HASH_TABLE 2 // Checking hash table (dwParam1 = current, dwParam2 = total)
#define CCB_COPYING_NON_MPQ_DATA 3 // Copying non-MPQ data: No params used
#define CCB_COMPACTING_FILES 4 // Compacting archive (dwParam1 = current, dwParam2 = total)
#define CCB_CLOSING_ARCHIVE 5 // Closing archive: No params used
#define LISTFILE_NAME "(listfile)" // Name of internal listfile
#define SIGNATURE_NAME "(signature)" // Name of internal signature
#define ATTRIBUTES_NAME "(attributes)" // Name of internal attributes file
#define STORMLIB_VERSION (0x0600) // Current version of StormLib
#define MPQ_FORMAT_VERSION_1 0 // Up to The Burning Crusade
#define MPQ_FORMAT_VERSION_2 1 // The Burning Crusade and newer,
// Flags for SFileOpenArchiveEx
#define MPQ_OPEN_NO_LISTFILE 0x00000001 // Don't add the internal listfile
// supports archives with size > 4 GB
// Additional flags for SFileCreateArchiveEx
#define MPQ_CREATE_ARCHIVE_V1 0x00000000 // Creates archive with size up to 4GB
#define MPQ_CREATE_ARCHIVE_V2 0x00010000 // Creates archive larger than 4 GB
//-----------------------------------------------------------------------------
// Structures
#if (defined(WIN32) || defined(WIN64))
#include <pshpack1.h>
#else
#pragma pack(1)
#endif
struct TMPQFile;
struct TMPQShunt
{
// The ID_MPQ_SHUNT ('MPQ\x1B') signature
DWORD dwID;
DWORD dwUnknown;
// Position of the MPQ header, relative to the begin of the shunt
DWORD dwHeaderPos;
};
// MPQ file header
struct TMPQHeader
{
// The ID_MPQ ('MPQ\x1A') signature
DWORD dwID;
// Size of the archive header
DWORD dwHeaderSize;
// Size of MPQ archive
// This field is deprecated in the Burning Crusade MoPaQ format, and the size of the archive
// is calculated as the size from the beginning of the archive to the end of the hash table,
// block table, or extended block table (whichever is largest).
DWORD dwArchiveSize;
// 0 = Original format
// 1 = Extended format (The Burning Crusade and newer)
USHORT wFormatVersion;
// Power of two exponent specifying the number of 512-byte disk sectors in each logical sector
// in the archive. The size of each logical sector in the archive is 512 * 2^SectorSizeShift.
// Bugs in the Storm library dictate that this should always be 3 (4096 byte sectors).
USHORT wBlockSize;
// Offset to the beginning of the hash table, relative to the beginning of the archive.
DWORD dwHashTablePos;
// Offset to the beginning of the block table, relative to the beginning of the archive.
DWORD dwBlockTablePos;
// Number of entries in the hash table. Must be a power of two, and must be less than 2^16 for
// the original MoPaQ format, or less than 2^20 for the Burning Crusade format.
DWORD dwHashTableSize;
// Number of entries in the block table
DWORD dwBlockTableSize;
};
// Extended MPQ file header. Valid only if wFormatVersion is 1 or higher
struct TMPQHeader2 : public TMPQHeader
{
// Offset to the beginning of the extended block table, relative to the beginning of the archive.
LARGE_INTEGER ExtBlockTablePos;
// High 16 bits of the hash table offset for large archives.
USHORT wHashTablePosHigh;
// High 16 bits of the block table offset for large archives.
USHORT wBlockTablePosHigh;
};
// Hash entry. All files in the archive are searched by their hashes.
struct TMPQHash
{
// The hash of the file path, using method A.
DWORD dwName1;
// The hash of the file path, using method B.
DWORD dwName2;
#ifdef PLATFORM_LITTLE_ENDIAN
// The language of the file. This is a Windows LANGID data type, and uses the same values.
// 0 indicates the default language (American English), or that the file is language-neutral.
USHORT lcLocale;
// The platform the file is used for. 0 indicates the default platform.
// No other values have been observed.
USHORT wPlatform;
#else
USHORT wPlatform;
USHORT lcLocale;
#endif
// If the hash table entry is valid, this is the index into the block table of the file.
// Otherwise, one of the following two values:
// - FFFFFFFFh: Hash table entry is empty, and has always been empty.
// Terminates searches for a given file.
// - FFFFFFFEh: Hash table entry is empty, but was valid at some point (a deleted file).
// Does not terminate searches for a given file.
DWORD dwBlockIndex;
};
// File description block contains informations about the file
struct TMPQBlock
{
// Offset of the beginning of the block, relative to the beginning of the archive.
DWORD dwFilePos;
// Compressed file size
DWORD dwCSize;
// Only valid if the block is a file; otherwise meaningless, and should be 0.
// If the file is compressed, this is the size of the uncompressed file data.
DWORD dwFSize;
// Flags for the file. See MPQ_FILE_XXXX constants
DWORD dwFlags;
};
// The extended block table was added to support archives larger than 4 gigabytes (2^32 bytes).
// The table contains the upper bits of the archive offsets for each block in the block table.
// It is simply an array of int16s, which become bits 32-47 of the archive offsets for each block,
// with bits 48-63 being zero. Individual blocks in the archive are still limited to 4 gigabytes
// in size. This table is only present in Burning Crusade format archives that exceed 4 gigabytes size.
struct TMPQBlockEx
{
USHORT wFilePosHigh;
};
struct TFileNode
{
DWORD dwRefCount; // Number of references
// There can be more files that have the same name.
// (e.g. multiple language files). We don't want to
// have an entry for each of them, so the entries will be referenced.
// When a number of node references reaches zero,
// the node will be deleted
size_t nLength; // File name length
char szFileName[1]; // File name, variable length
};
#if (defined(WIN32) || defined(WIN64))
#include <poppack.h>
#else
#pragma options align=reset
#endif
// Archive handle structure. Note that it does not agree with Storm.dll's structure.
struct TMPQArchive
{
// TMPQArchive * pNext; // Next archive (used by Storm.dll only)
// TMPQArchive * pPrev; // Previous archive (used by Storm.dll only)
char szFileName[MAX_PATH]; // Opened archive file name
HANDLE hFile; // File handle
DWORD dwPriority; // Priority of the archive
LARGE_INTEGER ShuntPos; // Position of MPQShunt (only valid if a shunt is present)
LARGE_INTEGER MpqPos; // MPQ position in the file, relative to the begin of the file
LARGE_INTEGER MpqSize; // Size of MPQ archive
LARGE_INTEGER HashTablePos; // Offset of the hast table in the MPQ, relative to the begin of the file
LARGE_INTEGER BlockTablePos; // Offset of the hast table in the MPQ, relative to the begin
LARGE_INTEGER ExtBlockTablePos; // Offset of the extended block table, relative to the begin
LARGE_INTEGER FilePointer; // Current position in the file (relative to begin of the file)
TMPQFile * pLastFile; // Recently read file
DWORD dwBlockPos; // Position of loaded block in the file
DWORD dwBlockSize; // Size of file block
BYTE * pbBlockBuffer; // Buffer (cache) for file block
DWORD dwBuffPos; // Position in block buffer
TMPQShunt * pShunt; // MPQ shunt (NULL if not present in the file)
TMPQHeader2 * pHeader; // MPQ file header
TMPQHash * pHashTable; // Hash table
TMPQBlock * pBlockTable; // Block table
TMPQBlockEx * pExtBlockTable; // Extended block table
TMPQShunt Shunt; // MPQ shunt. Valid only when ID_MPQ_SHUNT has been found
TMPQHeader2 Header; // MPQ header
// Non-Storm.dll members
TFileNode ** pListFile; // File name array
// HANDLE hListFile; // Handle to temporary listfile (when open with write access)
DWORD dwFlags; // See MPQ_FLAG_XXXXX
// BOOL bChanged; // TRUE if the archive was changed since open.
// BOOL bProtected; // TRUE if the archive is protected by somehow
};
// File handle structure. Note that it does not agree with Storm.dll structures
struct TMPQFile
{
HANDLE hFile; // File handle
TMPQArchive * ha; // Archive handle
TMPQHash * pHash; // Hash table entry
TMPQBlockEx * pBlockEx; // Pointer to extended file block entry
TMPQBlock * pBlock; // File block pointer
DWORD dwSeed1; // Seed used for file decrypt
DWORD dwFilePos; // Current file position
LARGE_INTEGER MpqFilePos; // Position of the file data in MPQ archive
// (relative to file begin)
DWORD * pdwBlockPos; // Position of each file block (only for compressed files)
DWORD nBlocks; // Number of blocks in the file (incl. the last noncomplete one)
BOOL bBlockPosLoaded; // TRUE if block positions loaded
BYTE * pbFileBuffer; // Decompressed file (for single unit files, size is the uncompressed file size)
DWORD dwHashIndex; // Index to Hash table
DWORD dwFileIndex; // Index to Block table
char szFileName[1]; // File name (variable length)
};
// Used by searching in MPQ archives
struct TMPQSearch
{
TMPQArchive * ha; // Handle to MPQ, where the search runs
DWORD dwNextIndex; // The next searched hash index
DWORD dwName1; // Lastly found Name1
DWORD dwName2; // Lastly found Name2
char szSearchMask[1]; // Search mask (variable length)
};
struct SFILE_FIND_DATA
{
char cFileName[MAX_PATH]; // Full name of the found file
char * szPlainName; // Pointer to file part
LCID lcLocale; // Locale version
DWORD dwFileSize; // File size in bytes
DWORD dwFileFlags; // File flags (compressed or encrypted)
DWORD dwBlockIndex; // Block index for the file
DWORD dwCompSize; // Compressed file size
};
//-----------------------------------------------------------------------------
// Memory management
//
// We use our own macros for allocating/freeing memory. If you want
// to redefine them, please keep the following rules
//
// - The memory allocation must return NULL if not enough memory
// (i.e not to throw exception)
// - It is not necessary to fill the allocated block with zeros
// - Memory freeing function must not test the pointer to NULL.
//
__inline void * DebugMalloc(char * szFile, int nLine, int nSize)
{
void * ptr = malloc(nSize + 100);
char * plain;
plain = strrchr(szFile, '\\');
if(plain == NULL)
plain = strrchr(szFile, '/');
if(plain == NULL)
plain = szFile;
#if _MSC_VER > 0x1300
sprintf_s((char *)ptr, nSize+100, "%s(%u)", plain, nLine);
#else
sprintf((char *)ptr, "%s(%u)", plain, nLine);
#endif
return (char *)ptr + 100;
}
__inline void DebugFree(void * ptr)
{
free((char *)ptr - 100);
}
#ifndef ALLOCMEM
#define ALLOCMEM(type, nitems) (type *)malloc((nitems) * sizeof(type))
#define FREEMEM(ptr) free(ptr)
#endif
//#define ALLOCMEM(type, nitems) (type *)DebugMalloc(__FILE__, __LINE__, (nitems) * sizeof(type))
//#define FREEMEM(ptr) DebugFree(ptr)
//-----------------------------------------------------------------------------
// Functions in StormLib - compatible with Storm.dll
// Typedefs for functions exported by Storm.dll
typedef LCID (WINAPI * SFILESETLOCALE)(LCID);
typedef BOOL (WINAPI * SFILEOPENARCHIVE)(const char *, DWORD, DWORD, HANDLE *);
typedef BOOL (WINAPI * SFILECLOSEARCHIVE)(HANDLE);
typedef BOOL (WINAPI * SFILEOPENFILEEX)(HANDLE, const char *, DWORD, HANDLE *);
typedef BOOL (WINAPI * SFILECLOSEFILE)(HANDLE);
typedef DWORD (WINAPI * SFILEGETFILESIZE)(HANDLE, DWORD *);
typedef DWORD (WINAPI * SFILESETFILEPOINTER)(HANDLE, LONG, LONG *, DWORD);
typedef BOOL (WINAPI * SFILEREADFILE)(HANDLE, VOID *, DWORD, DWORD *, LPOVERLAPPED);
// Archive opening/closing
LCID WINAPI SFileSetLocale(LCID lcNewLocale);
LCID WINAPI SFileGetLocale();
BOOL WINAPI SFileOpenArchive(const char * szMpqName, DWORD dwPriority, DWORD dwFlags, HANDLE * phMPQ);
BOOL WINAPI SFileCloseArchive(HANDLE hMPQ);
// File opening/closing
BOOL WINAPI SFileOpenFileEx(HANDLE hMPQ, const char * szFileName, DWORD dwSearchScope, HANDLE * phFile);
BOOL WINAPI SFileCloseFile(HANDLE hFile);
// File I/O
DWORD WINAPI SFileGetFilePos(HANDLE hFile, DWORD * pdwFilePosHigh = NULL);
DWORD WINAPI SFileGetFileSize(HANDLE hFile, DWORD * pdwFileSizeHigh = NULL);
DWORD WINAPI SFileSetFilePointer(HANDLE hFile, LONG lFilePos, LONG * pdwFilePosHigh, DWORD dwMethod);
BOOL WINAPI SFileReadFile(HANDLE hFile, VOID * lpBuffer, DWORD dwToRead, DWORD * pdwRead = NULL, LPOVERLAPPED lpOverlapped = NULL);
BOOL WINAPI SFileExtractFile(HANDLE hMpq, const char * szToExtract, const char * szExtracted);
// Adds another listfile into MPQ. The currently added listfile(s) remain,
// so you can use this API to combining more listfiles.
// Note that this function is internally called by SFileFindFirstFile
int WINAPI SFileAddListFile(HANDLE hMpq, const char * szListFile);
//-----------------------------------------------------------------------------
// Functions in StormLib - not implemented in Storm.dll
// Archive creating and editing
BOOL WINAPI SFileCreateArchiveEx(const char * szMpqName, DWORD dwCreationDisposition, DWORD dwHashTableSize, HANDLE * phMPQ);
BOOL WINAPI SFileAddFile(HANDLE hMPQ, const char * szFileName, const char * szArchivedName, DWORD dwFlags);
BOOL WINAPI SFileAddWave(HANDLE hMPQ, const char * szFileName, const char * szArchivedName, DWORD dwFlags, DWORD dwQuality);
BOOL WINAPI SFileRemoveFile(HANDLE hMPQ, const char * szFileName, DWORD dwSearchScope = SFILE_OPEN_BY_INDEX);
BOOL WINAPI SFileRenameFile(HANDLE hMPQ, const char * szOldFileName, const char * szNewFileName);
BOOL WINAPI SFileSetFileLocale(HANDLE hFile, LCID lcNewLocale);
// Retrieving info about the file
BOOL WINAPI SFileHasFile(HANDLE hMPQ, char * szFileName);
BOOL WINAPI SFileGetFileName(HANDLE hFile, char * szFileName);
DWORD_PTR WINAPI SFileGetFileInfo(HANDLE hMpqOrFile, DWORD dwInfoType);
// File search
// Note that the SFileFindFirstFileEx has been removed. Use SListFileFindFirst/Next
HANDLE WINAPI SFileFindFirstFile(HANDLE hMPQ, const char * szMask, SFILE_FIND_DATA * lpFindFileData, const char * szListFile);
BOOL WINAPI SFileFindNextFile(HANDLE hFind, SFILE_FIND_DATA * lpFindFileData);
BOOL WINAPI SFileFindClose(HANDLE hFind);
// Listfile search
HANDLE SListFileFindFirstFile(HANDLE hMpq, const char * szListFile, const char * szMask, SFILE_FIND_DATA * lpFindFileData);
BOOL SListFileFindNextFile(HANDLE hFind, SFILE_FIND_DATA * lpFindFileData);
BOOL SListFileFindClose(HANDLE hFind);
// Archive compacting
typedef void (WINAPI * COMPACTCB)(void * lpUserData, DWORD dwWorkType, DWORD dwParam1, DWORD dwParam2);
BOOL WINAPI SFileSetCompactCallback(HANDLE hMPQ, COMPACTCB CompactCB, void * lpData);
BOOL WINAPI SFileCompactArchive(HANDLE hMPQ, const char * szListFile = NULL, BOOL bReserved = 0);
// Locale support
int WINAPI SFileEnumLocales(HANDLE hMPQ, const char * szFileName, LCID * plcLocales, DWORD * pdwMaxLocales, DWORD dwSearchScope = SFILE_OPEN_BY_INDEX);
// (De)compression
int WINAPI SCompCompress (char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength, int uCompressions, int nCmpType, int nCmpLevel);
int WINAPI SCompDecompress (char * pbOutBuffer, int * pdwOutLength, char * pbInBuffer, int dwInLength);
// Sets the default data compression for files added to MPQ,
// if MPQ_FILE_COMPRESS_MULTI has been specified in call to SFileAddFile
int WINAPI SCompSetDataCompression(int nDataCompression);
//-----------------------------------------------------------------------------
// Functions from Storm.dll. They use slightly different names for keeping
// possibility to use them together with StormLib (StormXXX instead of SFileXXX)
#ifdef __LINK_STORM_DLL__
#define STORM_ALTERNATE_NAMES // Force Storm.h to use alternate fnc names
#include "StormDll.h"
#endif // __LINK_STORM_DLL__
//-----------------------------------------------------------------------------
// GFX decode functions. See GfxDecode.cpp for details and description
USHORT WINAPI celGetFrameCount(BYTE * fileBuf);
BYTE * WINAPI celGetFrameData(BYTE *fileBuf, BYTE *palette, USHORT xsize, USHORT frame, USHORT *ysize, USHORT *maxX=NULL);
USHORT WINAPI cl2GetFrameCount(BYTE *fileBuf);
BYTE ** WINAPI cl2GetDirData(BYTE *fileBuf, BYTE *palette, USHORT xsize, USHORT dir, USHORT *ysize);
BYTE * WINAPI pcxGetData(BYTE *filebuf, DWORD filesize, BYTE transcol, USHORT *xsize, USHORT *ysize);
#endif // __STORMLIB_H_

278
contrib/vmap_extractor_v2/stormlib/StormPort.h
View file

@ -1,278 +0,0 @@
/*****************************************************************************/
/* StormPort.h Copyright (c) Marko Friedemann 2001 */
/*---------------------------------------------------------------------------*/
/* Portability module for the StormLib library. Contains a wrapper symbols */
/* to make the compilation under Linux work */
/* */
/* Author: Marko Friedemann <marko.friedemann@bmx-chemnitz.de> */
/* Created at: Mon Jan 29 18:26:01 CEST 2001 */
/* Computer: whiplash.flachland-chemnitz.de */
/* System: Linux 2.4.0 on i686 */
/* */
/* Author: Sam Wilkins */
/* System: Mac OS X and port to big endian processor */
/* */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 29.01.01 1.00 Mar Created */
/* 24.03.03 1.01 Lad Some cosmetic changes */
/* 12.11.03 1.02 Dan Macintosh compatibility */
/* 24.07.04 1.03 Sam Mac OS X compatibility */
/* 22.11.06 1.04 Sam Mac OS X compatibility (for StormLib 6.0) */
/* 31.12.06 1.05 XPinguin Full GNU/Linux compatibility */
/*****************************************************************************/
#ifndef __STORMPORT_H__
#define __STORMPORT_H__
// Defines for Windows
#if !defined(PLATFORM_DEFINED) && (defined(WIN32) || defined(WIN64))
// In MSVC 8.0, there are some functions declared as deprecated.
#if _MSC_VER >= 1400
#define _CRT_SECURE_NO_DEPRECATE
#define _CRT_NON_CONFORMING_SWPRINTFS
#endif
#include <assert.h>
#include <stdio.h>
#include <windows.h>
#define PLATFORM_LITTLE_ENDIAN 1
#ifdef WIN64
#define PLATFORM_64BIT
#else
#define PLATFORM_32BIT
#endif
#define PLATFORM_DEFINED // The platform is known now
#endif
// Defines for Mac Carbon
#if !defined(PLATFORM_DEFINED) && defined(__APPLE__) // Mac Carbon API
// Macintosh using Carbon
#include <Carbon/Carbon.h> // Mac OS X
#define _stricmp strcasecmp // Case insensitive strcmp has a different name on this platform.
#define _strnicmp strncasecmp
typedef void * LPCSTR;
typedef unsigned long * LPDWORD;
typedef long * PLONG;
typedef void * LPVOID;
typedef unsigned int UINT;
#define PKEXPORT
#define __SYS_ZLIB
#define __SYS_BZLIB
#define LANG_NEUTRAL 0
#if defined(__BIG_ENDIAN__)
#define PLATFORM_LITTLE_ENDIAN 0
#else
#define PLATFORM_LITTLE_ENDIAN 1 // Apple is now making Macs with Intel CPUs
#endif
#define PLATFORM_DEFINED // The platform is known now
#endif
// Assumption: we are not on Windows nor Macintosh, so this must be linux *grin*
// Ladik : Why the hell Linux does not use some OS-dependent #define ?
#if !defined(PLATFORM_DEFINED)
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>
#include <stdlib.h>
#include <stdio.h>
#include <stdarg.h>
#include <string.h>
#include <ctype.h>
#include <assert.h>
#define PLATFORM_LITTLE_ENDIAN 1
#define PLATFORM_DEFINED
#define LANG_NEUTRAL 0
#endif /* not __powerc */
#if !defined(WIN32) && !defined(WIN64)
// Typedefs for ANSI C
typedef unsigned char BYTE;
typedef short SHORT;
typedef unsigned short WORD;
typedef unsigned short USHORT;
typedef long LONG;
typedef unsigned long DWORD;
typedef unsigned long DWORD_PTR;
typedef long LONG_PTR;
typedef long long LONGLONG;
#ifndef __OBJC__
#define BOOL bool
#endif
typedef void * HANDLE;
typedef void * LPOVERLAPPED; // Unsupported on Linux
typedef char TCHAR;
typedef unsigned long LCID;
typedef void * LPCSTR;
typedef unsigned long * LPDWORD;
typedef long * PLONG;
typedef void * LPVOID;
typedef unsigned int UINT;
typedef struct _FILETIME
{
DWORD dwLowDateTime;
DWORD dwHighDateTime;
}
FILETIME, *PFILETIME;
typedef union _LARGE_INTEGER
{
#if PLATFORM_LITTLE_ENDIAN
struct
{
DWORD LowPart;
LONG HighPart;
};
#else
struct
{
LONG HighPart;
DWORD LowPart;
};
#endif
LONGLONG QuadPart;
}
LARGE_INTEGER, *PLARGE_INTEGER;
// Some Windows-specific defines
#ifndef MAX_PATH
#define MAX_PATH 1024
#endif
#ifndef TRUE
#define TRUE true
#endif
#ifndef FALSE
#define FALSE false
#endif
#define VOID void
#define WINAPI
#define FILE_BEGIN SEEK_SET
#define FILE_CURRENT SEEK_CUR
#define FILE_END SEEK_END
#define CREATE_NEW 1
#define CREATE_ALWAYS 2
#define OPEN_EXISTING 3
#define OPEN_ALWAYS 4
#define FILE_SHARE_READ 0x00000001L
#define GENERIC_WRITE 0x40000000
#define GENERIC_READ 0x80000000
#define FILE_FLAG_DELETE_ON_CLOSE 1 // Sam: Added these two defines so it would compile.
#define FILE_FLAG_SEQUENTIAL_SCAN 2
#define ERROR_SUCCESS 0
#define ERROR_INVALID_FUNCTION 1
#define ERROR_FILE_NOT_FOUND 2
#define ERROR_ACCESS_DENIED 5
#define ERROR_NOT_ENOUGH_MEMORY 8
#define ERROR_BAD_FORMAT 11
#define ERROR_NO_MORE_FILES 18
#define ERROR_GEN_FAILURE 31
#define ERROR_HANDLE_EOF 38
#define ERROR_HANDLE_DISK_FULL 39
#define ERROR_NOT_SUPPORTED 50
#define ERROR_INVALID_PARAMETER 87
#define ERROR_DISK_FULL 112
#define ERROR_CALL_NOT_IMPLEMENTED 120
#define ERROR_ALREADY_EXISTS 183
#define ERROR_CAN_NOT_COMPLETE 1003
#define ERROR_PARAMETER_QUOTA_EXCEEDED 1283
#define ERROR_FILE_CORRUPT 1392
#define ERROR_INSUFFICIENT_BUFFER 4999
#define INVALID_HANDLE_VALUE ((HANDLE) -1)
#ifndef min
#define min(a, b) ((a < b) ? a : b)
#endif
#ifndef max
#define max(a, b) ((a > b) ? a : b)
#endif
#define _stricmp strcasecmp
#define _strnicmp strncasecmp
extern int globalerr;
void SetLastError(int err);
int GetLastError();
char *ErrString(int err);
// Emulation of functions for file I/O available in Win32
HANDLE CreateFile(const char * lpFileName, DWORD dwDesiredAccess, DWORD dwShareMode, void * lpSecurityAttributes, DWORD dwCreationDisposition, DWORD dwFlagsAndAttributes, HANDLE hTemplateFile);
BOOL CloseHandle(HANDLE hObject);
DWORD GetFileSize(HANDLE hFile, DWORD * lpFileSizeHigh);
DWORD SetFilePointer(HANDLE, LONG lDistanceToMove, LONG * lpDistanceToMoveHigh, DWORD dwMoveMethod);
BOOL SetEndOfFile(HANDLE hFile);
BOOL ReadFile(HANDLE hFile, void * lpBuffer, DWORD nNumberOfBytesToRead, DWORD * lpNumberOfBytesRead, void * lpOverLapped);
BOOL WriteFile(HANDLE hFile, const void * lpBuffer, DWORD nNumberOfBytesToWrite, DWORD * lpNumberOfBytesWritten, void * lpOverLapped);
BOOL IsBadReadPtr(const void * ptr, int size);
DWORD GetFileAttributes(const char * szileName);
BOOL DeleteFile(const char * lpFileName);
BOOL MoveFile(const char * lpFromFileName, const char * lpToFileName);
void GetTempPath(DWORD szTempLength, char * szTemp);
void GetTempFileName(const char * lpTempFolderPath, const char * lpFileName, DWORD something, char * szLFName);
#define strnicmp strncasecmp
#endif // !WIN32
#if PLATFORM_LITTLE_ENDIAN
#define BSWAP_INT16_UNSIGNED(a) (a)
#define BSWAP_INT16_SIGNED(a) (a)
#define BSWAP_INT32_UNSIGNED(a) (a)
#define BSWAP_INT32_SIGNED(a) (a)
#define BSWAP_ARRAY16_UNSIGNED(a,b) {}
#define BSWAP_ARRAY32_UNSIGNED(a,b) {}
#define BSWAP_TMPQSHUNT(a) {}
#define BSWAP_TMPQHEADER(a) {}
#else
extern unsigned short SwapUShort(unsigned short);
extern unsigned long SwapULong(unsigned long);
extern short SwapShort(unsigned short);
extern long SwapLong(unsigned long);
extern void ConvertUnsignedLongBuffer(unsigned long *buffer, unsigned long nbLongs);
extern void ConvertUnsignedShortBuffer(unsigned short *buffer, unsigned long nbShorts);
extern void ConvertTMPQShunt(void *shunt);
extern void ConvertTMPQHeader(void *header);
#define BSWAP_INT16_UNSIGNED(a) SwapUShort((a))
#define BSWAP_INT32_UNSIGNED(a) SwapULong((a))
#define BSWAP_INT16_SIGNED(a) SwapShort((a))
#define BSWAP_INT32_SIGNED(a) SwapLong((a))
#define BSWAP_ARRAY16_UNSIGNED(a,b) ConvertUnsignedShortBuffer((a),(b))
#define BSWAP_ARRAY32_UNSIGNED(a,b) ConvertUnsignedLongBuffer((a),(b))
#define BSWAP_TMPQSHUNT(a) ConvertTMPQShunt((a))
#define BSWAP_TMPQHEADER(a) ConvertTMPQHeader((a))
#endif
#endif // __STORMPORT_H__

168
contrib/vmap_extractor_v2/stormlib/StormPortLinux.cpp
View file

@ -1,168 +0,0 @@
/********************************************************************
*
* Description: implementation for StormLib - linux port
* intended to be used in GLdiablo
*
* ----> StormLib was originally developed for Windows by
* Ladislav Zezula (www.zezula.net), and he did
* a _great_ job! Thanks Ladislav!
*
* this is currently a quick and dirty hack to get it working
* don't expect beauty and/or miracles :)
*
* these are function wraps to execute Windows API calls
* as native Macintosh file calls (open/close/read/write/...)
*
* continue you work: added some wrapping functions for GNU/Linux by XPinguin
*
* Author: Marko Friedemann <marko.friedemann@bmx-chemnitz.de>
* Created at: Mon Jan 29 19:01:37 CEST 2001
* Computer: whiplash.flachland-chemnitz.de
* System: Linux 2.4.0 on i686
*
* Copyright (c) 2001 BMX-Chemnitz.DE All rights reserved.
*
********************************************************************/
#ifndef _WIN32
#include "StormPort.h"
int globalerr;
void SetLastError(int err)
{
globalerr = err;
}
int GetLastError()
{
return(globalerr);
}
char *ErrString(int err)
{
switch (err) {
case ERROR_INVALID_FUNCTION:
return "function not implemented";
case ERROR_FILE_NOT_FOUND:
return "file not found";
case ERROR_ACCESS_DENIED:
return "access denied";
case ERROR_NOT_ENOUGH_MEMORY:
return "not enough memory";
case ERROR_BAD_FORMAT:
return "bad format";
case ERROR_NO_MORE_FILES:
return "no more files";
case ERROR_HANDLE_EOF:
return "access beyound EOF";
case ERROR_HANDLE_DISK_FULL:
return "no space left on device";
case ERROR_INVALID_PARAMETER:
return "invalid parameter";
case ERROR_DISK_FULL:
return "no space left on device";
case ERROR_ALREADY_EXISTS:
return "file exists";
case ERROR_CAN_NOT_COMPLETE:
return "operation cannot be completed";
default:
return "unknown error";
}
}
HANDLE CreateFile(const char *sFileName, DWORD ulMode, DWORD ulSharing, void *pSecAttrib, DWORD ulCreation, DWORD ulFlags, HANDLE hFile)
{
switch (ulCreation) {
case OPEN_EXISTING:
return (HANDLE)open(sFileName, O_RDONLY | O_LARGEFILE);
case OPEN_ALWAYS:
return (HANDLE)open(sFileName, O_RDWR | O_CREAT);
case CREATE_NEW:
return (HANDLE)open(sFileName, O_RDWR | O_CREAT | O_TRUNC);
default:
return INVALID_HANDLE_VALUE;
}
}
BOOL CloseHandle(HANDLE hFile)
{
return (close((int)hFile) == 0);
}
DWORD GetFileSize(HANDLE hFile, DWORD *ulOffSetHigh)
{
if ((hFile == NULL) || (hFile == INVALID_HANDLE_VALUE))
return 0xffffffff;
struct stat fileinfo;
fstat((int)hFile, &fileinfo);
return fileinfo.st_size;
}
DWORD SetFilePointer(HANDLE hFile, LONG lOffSetLow, LONG *pOffSetHigh, DWORD ulMethod)
{
return lseek64((int)hFile, (off64_t)(*pOffSetHigh) << 32 | (DWORD)lOffSetLow, ulMethod);
}
BOOL SetEndOfFile(HANDLE hFile)
{
return (ftruncate((int)hFile, lseek((int)hFile, 0, SEEK_CUR)) == 0);
}
BOOL ReadFile(HANDLE hFile, void *pBuffer, DWORD ulLen, DWORD *ulRead, void *pOverLapped)
{
ssize_t count;
if ((count = read((int)hFile, pBuffer, ulLen)) == -1) {
*ulRead = 0;
return false;
}
*ulRead = count;
return true;
}
BOOL WriteFile(HANDLE hFile, const void *pBuffer, DWORD ulLen, DWORD *ulWritten, void *pOverLapped)
{
ssize_t count;
if ((count = write((int)hFile, pBuffer, ulLen)) == -1) {
*ulWritten = 0;
return false;
}
*ulWritten = count;
return true;
}
// Check if a memory block is accessible for reading
BOOL IsBadReadPtr(const void * ptr, int size)
{
return FALSE;
}
// Returns attributes of a file
DWORD GetFileAttributes(const char * szFileName)
{
return 0;
}
void GetTempPath(DWORD szTempLength, char * szTemp)
{
strncpy(szTemp, P_tmpdir, szTempLength);
}
void GetTempFileName(const char * lpTempFolderPath, const char * lpFileName, DWORD something, char * szLFName)
{
strcpy(szLFName, tempnam(lpTempFolderPath, lpFileName));
}
BOOL DeleteFile(const char *lpFileName)
{
return (BOOL)remove(lpFileName);
}
BOOL MoveFile(const char *lpExistingFileName, const char *lpNewFileName)
{
return rename(lpExistingFileName, lpNewFileName);
}
#endif

762
contrib/vmap_extractor_v2/stormlib/StormPortMac.cpp
View file

@ -1,762 +0,0 @@
/********************************************************************
*
* Description: implementation for StormLib - Macintosh port
*
* these are function wraps to execute Windows API calls
* as native Macintosh file calls (open/close/read/write/...)
* requires Mac OS X
*
* Derived from Marko Friedemann <marko.friedemann@bmx-chemnitz.de>
* StormPort.cpp for Linux
*
* Author: Daniel Chiaramello <daniel@chiaramello.net>
*
* Carbonized by: Sam Wilkins <swilkins1337@gmail.com>
*
********************************************************************/
#ifndef _WIN32
#include "StormPort.h"
#include "StormLib.h"
// FUNCTIONS EXTRACTED FROM MOREFILE PACKAGE!!!
// FEEL FREE TO REMOVE THEM AND TO ADD THE ORIGINAL ONES!
/*****************************************************************************
* BEGIN OF MOREFILES COPY-PASTE
*****************************************************************************/
#ifdef __USEPRAGMAINTERNAL
#ifdef __MWERKS__
#pragma internal on
#endif
#endif
union TLongAnd4Bytes
{
unsigned char bytes[4];
unsigned long uvalue;
signed long svalue;
};
static OSErr FSGetFullPath(const FSRef *ref, UInt8 *fullPath, UInt32 fullPathLength)
{
OSErr result;
result = FSRefMakePath(ref, fullPath, fullPathLength);
return result;
}
static OSErr FSLocationFromFullPath(const void *fullPath, FSRef *ref)
{
OSErr result;
result = FSPathMakeRef((UInt8 *)fullPath, ref, NULL); // Create an FSRef from the path
return result;
}
/*****************************************************************************/
/*****************************************************************************/
static OSErr FSCreateCompat(const FSRef *parentRef, OSType creator, OSType fileType, const UniChar *fileName,
UniCharCount nameLength, FSRef *ref)
{
FSCatalogInfo theCatInfo;
OSErr theErr;
((FileInfo *)&theCatInfo.finderInfo)->fileCreator = creator;
((FileInfo *)&theCatInfo.finderInfo)->fileType = fileType;
((FileInfo *)&theCatInfo.finderInfo)->finderFlags = 0;
SetPt(&((FileInfo *)&theCatInfo.finderInfo)->location, 0, 0);
((FileInfo *)&theCatInfo.finderInfo)->reservedField = 0;
theErr = FSCreateFileUnicode(parentRef, nameLength, fileName, kFSCatInfoFinderInfo, &theCatInfo, ref, NULL);
return theErr;
}
/*****************************************************************************/
static OSErr FSOpenDFCompat(FSRef *ref, char permission, short *refNum)
{
HFSUniStr255 forkName;
OSErr theErr;
Boolean isFolder, wasChanged;
theErr = FSResolveAliasFile(ref, TRUE, &isFolder, &wasChanged);
if (theErr != noErr)
return theErr;
FSGetDataForkName(&forkName);
theErr = FSOpenFork(ref, forkName.length, forkName.unicode, permission, refNum);
return theErr;
}
/*****************************************************************************
* END OF MOREFILES COPY-PASTE
*****************************************************************************/
#pragma mark -
int globalerr;
/********************************************************************
* SwapLong
********************************************************************/
unsigned long SwapULong(unsigned long data)
{
// Apple provided function
uint32_t result;
__asm__("lwbrx %0,0,%1" : "=r" (result) : "r" (&data), "m" (data));
return result;
/*
TLongAnd4Bytes Work;
unsigned char * value_as_4bytes = (unsigned char *)&value;
Work.bytes[0] = value_as_4bytes[3];
Work.bytes[1] = value_as_4bytes[2];
Work.bytes[2] = value_as_4bytes[1];
Work.bytes[3] = value_as_4bytes[0];
return Work.uvalue;
*/
}
long SwapLong(unsigned long data)
{
// Apple provided function
uint32_t result;
__asm__("lwbrx %0,0,%1" : "=r" (result) : "r" (&data), "m" (data));
return (long)result;
/*
TLongAnd4Bytes Work;
unsigned char * value_as_4bytes = (unsigned char *)&value;
Work.bytes[0] = value_as_4bytes[3];
Work.bytes[1] = value_as_4bytes[2];
Work.bytes[2] = value_as_4bytes[1];
Work.bytes[3] = value_as_4bytes[0];
return Work.svalue;
*/
}
/********************************************************************
* SwapShort
********************************************************************/
unsigned short SwapUShort(unsigned short data)
{
// Apple provided function
uint16_t result;
__asm__("lhbrx %0,0,%1" : "=r" (result) : "r" (&data), "m" (data));
return result;
}
short SwapShort(unsigned short data)
{
// Apple provided function
uint16_t result;
__asm__("lhbrx %0,0,%1" : "=r" (result) : "r" (&data), "m" (data));
return (short)result;
}
/********************************************************************
* ConvertUnsignedLongBuffer
********************************************************************/
void ConvertUnsignedLongBuffer(unsigned long *buffer, unsigned long nbLongs)
{
while (nbLongs-- > 0)
{
*buffer = SwapLong(*buffer);
buffer++;
}
}
/********************************************************************
* ConvertUnsignedShortBuffer
********************************************************************/
void ConvertUnsignedShortBuffer(unsigned short *buffer, unsigned long nbShorts)
{
while (nbShorts-- > 0)
{
*buffer = SwapShort(*buffer);
buffer++;
}
}
/********************************************************************
* ConvertTMPQShunt
********************************************************************/
void ConvertTMPQShunt(void *shunt)
{
TMPQShunt * theShunt = (TMPQShunt *)shunt;
theShunt->dwID = SwapULong(theShunt->dwID);
theShunt->dwUnknown = SwapULong(theShunt->dwUnknown);
theShunt->dwHeaderPos = SwapULong(theShunt->dwHeaderPos);
}
/********************************************************************
* ConvertTMPQHeader
********************************************************************/
void ConvertTMPQHeader(void *header)
{
TMPQHeader2 * theHeader = (TMPQHeader2 *)header;
theHeader->dwID = SwapULong(theHeader->dwID);
theHeader->dwHeaderSize = SwapULong(theHeader->dwHeaderSize);
theHeader->dwArchiveSize = SwapULong(theHeader->dwArchiveSize);
theHeader->wFormatVersion = SwapUShort(theHeader->wFormatVersion);
theHeader->wBlockSize = SwapUShort(theHeader->wBlockSize);
theHeader->dwHashTablePos = SwapULong(theHeader->dwHashTablePos);
theHeader->dwBlockTablePos = SwapULong(theHeader->dwBlockTablePos);
theHeader->dwHashTableSize = SwapULong(theHeader->dwHashTableSize);
theHeader->dwBlockTableSize = SwapULong(theHeader->dwBlockTableSize);
if(theHeader->wFormatVersion >= MPQ_FORMAT_VERSION_2)
{
DWORD dwTemp = theHeader->ExtBlockTablePos.LowPart;
theHeader->ExtBlockTablePos.LowPart = theHeader->ExtBlockTablePos.HighPart;
theHeader->ExtBlockTablePos.HighPart = dwTemp;
theHeader->ExtBlockTablePos.LowPart = SwapULong(theHeader->ExtBlockTablePos.LowPart);
theHeader->ExtBlockTablePos.HighPart = SwapULong(theHeader->ExtBlockTablePos.HighPart);
theHeader->wHashTablePosHigh = SwapUShort(theHeader->wHashTablePosHigh);
theHeader->wBlockTablePosHigh = SwapUShort(theHeader->wBlockTablePosHigh);
}
}
/********************************************************************
* ConvertTMPQHash
********************************************************************/
void ConvertHashTable(void *hashtable, DWORD nHashEntries)
{
TMPQHash * theHash = (TMPQHash *)hashtable;
USHORT lcLocale;
for(DWORD i = 0; i < nHashEntries; i++, theHash++)
{
lcLocale = theHash->lcLocale;
theHash->lcLocale = theHash->wPlatform;
theHash->wPlatform = lcLocale;
}
}
#pragma mark -
/********************************************************************
* SetLastError
********************************************************************/
void SetLastError(int err)
{
globalerr = err;
}
/********************************************************************
* GetLastError
********************************************************************/
int GetLastError()
{
return globalerr;
}
/********************************************************************
* ErrString
********************************************************************/
char *ErrString(int err)
{
switch (err)
{
case ERROR_INVALID_FUNCTION:
return "function not implemented";
case ERROR_FILE_NOT_FOUND:
return "file not found";
case ERROR_ACCESS_DENIED:
return "access denied";
case ERROR_NOT_ENOUGH_MEMORY:
return "not enough memory";
case ERROR_BAD_FORMAT:
return "bad format";
case ERROR_NO_MORE_FILES:
return "no more files";
case ERROR_HANDLE_EOF:
return "access beyound EOF";
case ERROR_HANDLE_DISK_FULL:
return "no space left on device";
case ERROR_INVALID_PARAMETER:
return "invalid parameter";
case ERROR_DISK_FULL:
return "no space left on device";
case ERROR_ALREADY_EXISTS:
return "file exists";
case ERROR_CAN_NOT_COMPLETE:
return "operation cannot be completed";
case ERROR_INSUFFICIENT_BUFFER:
return "insufficient buffer";
default:
return "unknown error";
}
}
#pragma mark -
/********************************************************************
* GetTempPath - returns a '/' or ':'-terminated path
* szTempLength: length for path
* szTemp: file path
********************************************************************/
void GetTempPath(DWORD szTempLength, char * szTemp) // I think I'll change this to use FSRefs.
{
FSRef theFSRef;
OSErr theErr = FSFindFolder(kOnAppropriateDisk, kTemporaryFolderType, kCreateFolder, &theFSRef);
if (theErr == noErr)
{
theErr = FSGetFullPath(&theFSRef, (UInt8 *)szTemp, MAX_PATH);
if (theErr != noErr)
szTemp[0] = '\0';
}
else
szTemp[0] = '\0';
strcat(szTemp, "/");
SetLastError(theErr);
}
/********************************************************************
* GetTempFileName
* lpTempFolderPath: the temporary folder path, terminated by "/"
* lpFileName: a file name base
* something: unknown
* szLFName: the final path, built from the path, the file name and a random pattern
********************************************************************/
void GetTempFileName(const char * lpTempFolderPath, const char * lpFileName, DWORD something, char * szLFName)
{
#pragma unused (something)
char tmp[2] = "A";
while (true)
{
HANDLE fHandle;
strcpy(szLFName, lpTempFolderPath);
strcat(szLFName, lpFileName);
strcat(szLFName, tmp);
if ((fHandle = CreateFile(szLFName, GENERIC_READ, 0, NULL, OPEN_EXISTING, 0, 0)) == INVALID_HANDLE_VALUE)
// OK we found it!
break;
CloseHandle(fHandle);
tmp[0]++;
}
}
/********************************************************************
* DeleteFile
* lpFileName: file path
********************************************************************/
BOOL DeleteFile(const char * lpFileName)
{
OSErr theErr;
FSRef theFileRef;
theErr = FSLocationFromFullPath(lpFileName, &theFileRef);
if (theErr != noErr)
{
SetLastError(theErr);
return FALSE;
}
theErr = FSDeleteObject(&theFileRef);
SetLastError(theErr);
return theErr == noErr;
}
/********************************************************************
* MoveFile
* lpFromFileName: old file path
* lpToFileName: new file path
********************************************************************/
BOOL MoveFile(const char * lpFromFileName, const char * lpToFileName)
{
OSErr theErr;
FSRef fromFileRef;
FSRef toFileRef;
FSRef parentFolderRef;
// Get the path to the old file
theErr = FSLocationFromFullPath(lpFromFileName, &fromFileRef);
if (theErr != noErr)
{
SetLastError(theErr);
return false;
}
// Get the path to the new folder for the file
char folderName[strlen(lpToFileName)];
CFStringRef folderPathCFString = CFStringCreateWithCString(NULL, lpToFileName, kCFStringEncodingUTF8);
CFURLRef fileURL = CFURLCreateWithFileSystemPath(NULL, folderPathCFString, kCFURLPOSIXPathStyle, FALSE);
CFURLRef folderURL = CFURLCreateCopyDeletingLastPathComponent(NULL, fileURL);
CFURLGetFileSystemRepresentation(folderURL, TRUE, (UInt8 *)folderName, strlen(lpToFileName));
theErr = FSLocationFromFullPath(folderName, &parentFolderRef);
CFRelease(fileURL);
CFRelease(folderURL);
CFRelease(folderPathCFString);
// Move the old file
theErr = FSMoveObject(&fromFileRef, &parentFolderRef, &toFileRef);
if (theErr != noErr)
{
SetLastError(theErr);
return false;
}
// Get a CFString for the new file name
CFStringRef newFileNameCFString = CFStringCreateWithCString(NULL, lpToFileName, kCFStringEncodingUTF8);
fileURL = CFURLCreateWithFileSystemPath(NULL, newFileNameCFString, kCFURLPOSIXPathStyle, FALSE);
CFRelease(newFileNameCFString);
newFileNameCFString = CFURLCopyLastPathComponent(fileURL);
CFRelease(fileURL);
// Convert CFString to Unicode and rename the file
UniChar unicodeFileName[256];
CFStringGetCharacters(newFileNameCFString, CFRangeMake(0, CFStringGetLength(newFileNameCFString)),
unicodeFileName);
theErr = FSRenameUnicode(&toFileRef, CFStringGetLength(newFileNameCFString), unicodeFileName,
kTextEncodingUnknown, NULL);
if (theErr != noErr)
{
SetLastError(theErr);
CFRelease(newFileNameCFString);
return false;
}
CFRelease(newFileNameCFString);
SetLastError(theErr);
return true;
}
/********************************************************************
* CreateFile
* ulMode: GENERIC_READ | GENERIC_WRITE
* ulSharing: FILE_SHARE_READ
* pSecAttrib: NULL
* ulCreation: OPEN_EXISTING, OPEN_ALWAYS, CREATE_NEW
* ulFlags: 0
* hFile: NULL
********************************************************************/
HANDLE CreateFile( const char *sFileName, /* file name */
DWORD ulMode, /* access mode */
DWORD ulSharing, /* share mode */
void *pSecAttrib, /* SD */
DWORD ulCreation, /* how to create */
DWORD ulFlags, /* file attributes */
HANDLE hFile ) /* handle to template file */
{
#pragma unused (ulSharing, pSecAttrib, ulFlags, hFile)
OSErr theErr;
FSRef theFileRef;
FSRef theParentRef;
short fileRef;
char permission;
static OSType gCreator;
static OSType gType;
theErr = FSLocationFromFullPath(sFileName, &theFileRef);
if (theErr == fnfErr)
{ // Create the FSRef for the parent directory.
memset(&theFileRef, 0, sizeof(FSRef));
UInt8 folderName[MAX_PATH];
CFStringRef folderPathCFString = CFStringCreateWithCString(NULL, sFileName, kCFStringEncodingUTF8);
CFURLRef fileURL = CFURLCreateWithFileSystemPath(NULL, folderPathCFString, kCFURLPOSIXPathStyle, FALSE);
CFURLRef folderURL = CFURLCreateCopyDeletingLastPathComponent(NULL, fileURL);
CFURLGetFileSystemRepresentation(folderURL, TRUE, folderName, MAX_PATH);
theErr = FSLocationFromFullPath(folderName, &theParentRef);
CFRelease(fileURL);
CFRelease(folderURL);
CFRelease(folderPathCFString);
}
if (theErr != noErr)
{
SetLastError(theErr);
if (ulCreation == OPEN_EXISTING || theErr != fnfErr)
return INVALID_HANDLE_VALUE;
}
if (ulCreation != OPEN_EXISTING)
{ /* We create the file */
UniChar unicodeFileName[256];
CFStringRef filePathCFString = CFStringCreateWithCString(NULL, sFileName, kCFStringEncodingUTF8);
CFURLRef fileURL = CFURLCreateWithFileSystemPath(NULL, filePathCFString, kCFURLPOSIXPathStyle, FALSE);
CFStringRef fileNameCFString = CFURLCopyLastPathComponent(fileURL);
CFStringGetCharacters(fileNameCFString, CFRangeMake(0, CFStringGetLength(fileNameCFString)),
unicodeFileName);
theErr = FSCreateCompat(&theParentRef, gCreator, gType, unicodeFileName,
CFStringGetLength(fileNameCFString), &theFileRef);
CFRelease(fileNameCFString);
CFRelease(filePathCFString);
CFRelease(fileURL);
if (theErr != noErr)
{
SetLastError(theErr);
return INVALID_HANDLE_VALUE;
}
}
if (ulMode == GENERIC_READ)
permission = fsRdPerm;
else
{
if (ulMode == GENERIC_WRITE)
permission = fsWrPerm;
else
permission = fsRdWrPerm;
}
theErr = FSOpenDFCompat(&theFileRef, permission, &fileRef);
SetLastError(theErr);
if (theErr == noErr)
return (HANDLE)(int)fileRef;
else
return INVALID_HANDLE_VALUE;
}
/********************************************************************
* CloseHandle
********************************************************************/
BOOL CloseHandle( HANDLE hFile ) /* handle to object */
{
OSErr theErr;
if ((hFile == NULL) || (hFile == INVALID_HANDLE_VALUE))
return FALSE;
theErr = FSCloseFork((short)(int)hFile);
SetLastError(theErr);
return theErr != noErr;
}
/********************************************************************
* GetFileSize
********************************************************************/
DWORD GetFileSize( HANDLE hFile, /* handle to file */
DWORD *ulOffSetHigh ) /* high-order word of file size */
{
SInt64 fileLength;
OSErr theErr;
if ((hFile == NULL) || (hFile == INVALID_HANDLE_VALUE))
{
SetLastError(theErr);
return -1u;
}
theErr = FSGetForkSize((short)(int)hFile, &fileLength);
if (theErr != noErr)
{
SetLastError(theErr);
return -1u;
}
if (ulOffSetHigh != NULL)
*ulOffSetHigh = fileLength >> 32;
SetLastError(theErr);
return fileLength;
}
/********************************************************************
* SetFilePointer
* pOffSetHigh: NULL
* ulMethod: FILE_BEGIN, FILE_CURRENT
********************************************************************/
DWORD SetFilePointer( HANDLE hFile, /* handle to file */
LONG lOffSetLow, /* bytes to move pointer */
LONG *pOffSetHigh, /* bytes to move pointer */
DWORD ulMethod ) /* starting point */
{
OSErr theErr;
if (ulMethod == FILE_CURRENT)
{
SInt64 bytesToMove;
if (pOffSetHigh != NULL)
bytesToMove = ((SInt64)*pOffSetHigh << 32) + lOffSetLow;
else
bytesToMove = lOffSetLow;
SInt64 newPos;
theErr = FSSetForkPosition((short)(int)hFile, fsFromMark, bytesToMove);
if (theErr != noErr)
{
SetLastError(theErr);
return -1u;
}
theErr = FSGetForkPosition((short)(int)hFile, &newPos);
if (theErr != noErr)
{
SetLastError(theErr);
return -1u;
}
if (pOffSetHigh != NULL)
*pOffSetHigh = newPos >> 32;
SetLastError(theErr);
return newPos;
}
else if (ulMethod == FILE_BEGIN)
{
SInt64 bytesToMove;
if (pOffSetHigh != NULL)
bytesToMove = ((SInt64)*pOffSetHigh << 32) + lOffSetLow;
else
bytesToMove = lOffSetLow;
theErr = FSSetForkPosition((short)(int)hFile, fsFromStart, bytesToMove);
if (theErr != noErr)
{
SetLastError(theErr);
return -1u;
}
SetLastError(theErr);
return lOffSetLow;
}
else
{
SInt64 bytesToMove;
if (pOffSetHigh != NULL)
bytesToMove = ((SInt64)*pOffSetHigh << 32) + lOffSetLow;
else
bytesToMove = lOffSetLow;
SInt64 newPos;
theErr = FSSetForkPosition((short)(int)hFile, fsFromLEOF, bytesToMove);
if (theErr != noErr)
{
SetLastError(theErr);
return -1u;
}
theErr = FSGetForkPosition((short)(int)hFile, &newPos);
if (theErr != noErr)
{
SetLastError(theErr);
return -1u;
}
if (pOffSetHigh != NULL)
*pOffSetHigh = newPos >> 32;
SetLastError(theErr);
return newPos;
}
}
/********************************************************************
* SetEndOfFile
********************************************************************/
BOOL SetEndOfFile( HANDLE hFile ) /* handle to file */
{
OSErr theErr;
theErr = FSSetForkSize((short)(int)hFile, fsAtMark, 0);
SetLastError(theErr);
return theErr == noErr;
}
/********************************************************************
* ReadFile
* pOverLapped: NULL
********************************************************************/
BOOL ReadFile( HANDLE hFile, /* handle to file */
void *pBuffer, /* data buffer */
DWORD ulLen, /* number of bytes to read */
DWORD *ulRead, /* number of bytes read */
void *pOverLapped ) /* overlapped buffer */
{
#pragma unused (pOverLapped)
ByteCount nbCharsRead;
OSErr theErr;
nbCharsRead = ulLen;
theErr = FSReadFork((short)(int)hFile, fsAtMark, 0, nbCharsRead, pBuffer, &nbCharsRead);
*ulRead = nbCharsRead;
SetLastError(theErr);
return theErr == noErr;
}
/********************************************************************
* WriteFile
* pOverLapped: NULL
********************************************************************/
BOOL WriteFile( HANDLE hFile, /* handle to file */
const void *pBuffer, /* data buffer */
DWORD ulLen, /* number of bytes to write */
DWORD *ulWritten, /* number of bytes written */
void *pOverLapped ) /* overlapped buffer */
{
#pragma unused (pOverLapped)
ByteCount nbCharsToWrite;
OSErr theErr;
nbCharsToWrite = ulLen;
theErr = FSWriteFork((short)(int)hFile, fsAtMark, 0, nbCharsToWrite, pBuffer, &nbCharsToWrite);
*ulWritten = nbCharsToWrite;
SetLastError(theErr);
return theErr == noErr;
}
// Check if a memory block is accessible for reading. It's probably too
// hard to check on Mac, so sorry, we'll just have to crash
BOOL IsBadReadPtr(const void * ptr, int size)
{
#pragma unused (ptr, size)
return FALSE;
}
// Returns attributes of a file. Actually, it doesn't, it just checks if
// the file exists, since that's all StormLib uses it for
DWORD GetFileAttributes(const char * szFileName)
{
FSRef theRef;
OSErr theErr;
theErr = FSLocationFromFullPath(szFileName, &theRef);
if (theErr != noErr)
return -1u;
else
return 0;
}
#endif

275
contrib/vmap_extractor_v2/stormlib/bzip2/CHANGES
View file

@ -1,275 +0,0 @@
0.9.0
~~~~~
First version.
0.9.0a
~~~~~~
Removed 'ranlib' from Makefile, since most modern Unix-es
don't need it, or even know about it.
0.9.0b
~~~~~~
Fixed a problem with error reporting in bzip2.c. This does not effect
the library in any way. Problem is: versions 0.9.0 and 0.9.0a (of the
program proper) compress and decompress correctly, but give misleading
error messages (internal panics) when an I/O error occurs, instead of
reporting the problem correctly. This shouldn't give any data loss
(as far as I can see), but is confusing.
Made the inline declarations disappear for non-GCC compilers.
0.9.0c
~~~~~~
Fixed some problems in the library pertaining to some boundary cases.
This makes the library behave more correctly in those situations. The
fixes apply only to features (calls and parameters) not used by
bzip2.c, so the non-fixedness of them in previous versions has no
effect on reliability of bzip2.c.
In bzlib.c:
* made zero-length BZ_FLUSH work correctly in bzCompress().
* fixed bzWrite/bzRead to ignore zero-length requests.
* fixed bzread to correctly handle read requests after EOF.
* wrong parameter order in call to bzDecompressInit in
bzBuffToBuffDecompress. Fixed.
In compress.c:
* changed setting of nGroups in sendMTFValues() so as to
do a bit better on small files. This _does_ effect
bzip2.c.
0.9.5a
~~~~~~
Major change: add a fallback sorting algorithm (blocksort.c)
to give reasonable behaviour even for very repetitive inputs.
Nuked --repetitive-best and --repetitive-fast since they are
no longer useful.
Minor changes: mostly a whole bunch of small changes/
bugfixes in the driver (bzip2.c). Changes pertaining to the
user interface are:
allow decompression of symlink'd files to stdout
decompress/test files even without .bz2 extension
give more accurate error messages for I/O errors
when compressing/decompressing to stdout, don't catch control-C
read flags from BZIP2 and BZIP environment variables
decline to break hard links to a file unless forced with -f
allow -c flag even with no filenames
preserve file ownerships as far as possible
make -s -1 give the expected block size (100k)
add a flag -q --quiet to suppress nonessential warnings
stop decoding flags after --, so files beginning in - can be handled
resolved inconsistent naming: bzcat or bz2cat ?
bzip2 --help now returns 0
Programming-level changes are:
fixed syntax error in GET_LL4 for Borland C++ 5.02
let bzBuffToBuffDecompress return BZ_DATA_ERROR{_MAGIC}
fix overshoot of mode-string end in bzopen_or_bzdopen
wrapped bzlib.h in #ifdef __cplusplus ... extern "C" { ... }
close file handles under all error conditions
added minor mods so it compiles with DJGPP out of the box
fixed Makefile so it doesn't give problems with BSD make
fix uninitialised memory reads in dlltest.c
0.9.5b
~~~~~~
Open stdin/stdout in binary mode for DJGPP.
0.9.5c
~~~~~~
Changed BZ_N_OVERSHOOT to be ... + 2 instead of ... + 1. The + 1
version could cause the sorted order to be wrong in some extremely
obscure cases. Also changed setting of quadrant in blocksort.c.
0.9.5d
~~~~~~
The only functional change is to make bzlibVersion() in the library
return the correct string. This has no effect whatsoever on the
functioning of the bzip2 program or library. Added a couple of casts
so the library compiles without warnings at level 3 in MS Visual
Studio 6.0. Included a Y2K statement in the file Y2K_INFO. All other
changes are minor documentation changes.
1.0
~~~
Several minor bugfixes and enhancements:
* Large file support. The library uses 64-bit counters to
count the volume of data passing through it. bzip2.c
is now compiled with -D_FILE_OFFSET_BITS=64 to get large
file support from the C library. -v correctly prints out
file sizes greater than 4 gigabytes. All these changes have
been made without assuming a 64-bit platform or a C compiler
which supports 64-bit ints, so, except for the C library
aspect, they are fully portable.
* Decompression robustness. The library/program should be
robust to any corruption of compressed data, detecting and
handling _all_ corruption, instead of merely relying on
the CRCs. What this means is that the program should
never crash, given corrupted data, and the library should
always return BZ_DATA_ERROR.
* Fixed an obscure race-condition bug only ever observed on
Solaris, in which, if you were very unlucky and issued
control-C at exactly the wrong time, both input and output
files would be deleted.
* Don't run out of file handles on test/decompression when
large numbers of files have invalid magic numbers.
* Avoid library namespace pollution. Prefix all exported
symbols with BZ2_.
* Minor sorting enhancements from my DCC2000 paper.
* Advance the version number to 1.0, so as to counteract the
(false-in-this-case) impression some people have that programs
with version numbers less than 1.0 are in some way, experimental,
pre-release versions.
* Create an initial Makefile-libbz2_so to build a shared library.
Yes, I know I should really use libtool et al ...
* Make the program exit with 2 instead of 0 when decompression
fails due to a bad magic number (ie, an invalid bzip2 header).
Also exit with 1 (as the manual claims :-) whenever a diagnostic
message would have been printed AND the corresponding operation
is aborted, for example
bzip2: Output file xx already exists.
When a diagnostic message is printed but the operation is not
aborted, for example
bzip2: Can't guess original name for wurble -- using wurble.out
then the exit value 0 is returned, unless some other problem is
also detected.
I think it corresponds more closely to what the manual claims now.
1.0.1
~~~~~
* Modified dlltest.c so it uses the new BZ2_ naming scheme.
* Modified makefile-msc to fix minor build probs on Win2k.
* Updated README.COMPILATION.PROBLEMS.
There are no functionality changes or bug fixes relative to version
1.0.0. This is just a documentation update + a fix for minor Win32
build problems. For almost everyone, upgrading from 1.0.0 to 1.0.1 is
utterly pointless. Don't bother.
1.0.2
~~~~~
A bug fix release, addressing various minor issues which have appeared
in the 18 or so months since 1.0.1 was released. Most of the fixes
are to do with file-handling or documentation bugs. To the best of my
knowledge, there have been no data-loss-causing bugs reported in the
compression/decompression engine of 1.0.0 or 1.0.1.
Note that this release does not improve the rather crude build system
for Unix platforms. The general plan here is to autoconfiscate/
libtoolise 1.0.2 soon after release, and release the result as 1.1.0
or perhaps 1.2.0. That, however, is still just a plan at this point.
Here are the changes in 1.0.2. Bug-reporters and/or patch-senders in
parentheses.
* Fix an infinite segfault loop in 1.0.1 when a directory is
encountered in -f (force) mode.
(Trond Eivind Glomsrod, Nicholas Nethercote, Volker Schmidt)
* Avoid double fclose() of output file on certain I/O error paths.
(Solar Designer)
* Don't fail with internal error 1007 when fed a long stream (> 48MB)
of byte 251. Also print useful message suggesting that 1007s may be
caused by bad memory.
(noticed by Juan Pedro Vallejo, fixed by me)
* Fix uninitialised variable silly bug in demo prog dlltest.c.
(Jorj Bauer)
* Remove 512-MB limitation on recovered file size for bzip2recover
on selected platforms which support 64-bit ints. At the moment
all GCC supported platforms, and Win32.
(me, Alson van der Meulen)
* Hard-code header byte values, to give correct operation on platforms
using EBCDIC as their native character set (IBM's OS/390).
(Leland Lucius)
* Copy file access times correctly.
(Marty Leisner)
* Add distclean and check targets to Makefile.
(Michael Carmack)
* Parameterise use of ar and ranlib in Makefile. Also add $(LDFLAGS).
(Rich Ireland, Bo Thorsen)
* Pass -p (create parent dirs as needed) to mkdir during make install.
(Jeremy Fusco)
* Dereference symlinks when copying file permissions in -f mode.
(Volker Schmidt)
* Majorly simplify implementation of uInt64_qrm10.
(Bo Lindbergh)
* Check the input file still exists before deleting the output one,
when aborting in cleanUpAndFail().
(Joerg Prante, Robert Linden, Matthias Krings)
Also a bunch of patches courtesy of Philippe Troin, the Debian maintainer
of bzip2:
* Wrapper scripts (with manpages): bzdiff, bzgrep, bzmore.
* Spelling changes and minor enhancements in bzip2.1.
* Avoid race condition between creating the output file and setting its
interim permissions safely, by using fopen_output_safely().
No changes to bzip2recover since there is no issue with file
permissions there.
* do not print senseless report with -v when compressing an empty
file.
* bzcat -f works on non-bzip2 files.
* do not try to escape shell meta-characters on unix (the shell takes
care of these).
* added --fast and --best aliases for -1 -9 for gzip compatibility.
1.0.3 (15 Feb 05)
~~~~~~~~~~~~~~~~~
Fixes some minor bugs since the last version, 1.0.2.
* Further robustification against corrupted compressed data.
There are currently no known bitstreams which can cause the
decompressor to crash, loop or access memory which does not
belong to it. If you are using bzip2 or the library to
decompress bitstreams from untrusted sources, an upgrade
to 1.0.3 is recommended.
* The documentation has been converted to XML, from which html
and pdf can be derived.
* Various minor bugs in the documentation have been fixed.
* Fixes for various compilation warnings with newer versions of
gcc, and on 64-bit platforms.
* The BZ_NO_STDIO cpp symbol was not properly observed in 1.0.2.
This has been fixed.

40
contrib/vmap_extractor_v2/stormlib/bzip2/LICENSE
View file

@ -1,40 +0,0 @@
This program, "bzip2", the associated library "libbzip2", and all
documentation, are copyright (C) 1996-2005 Julian R Seward. All
rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@acm.org
bzip2/libbzip2 version 1.0.3 of 15 February 2005

44
contrib/vmap_extractor_v2/stormlib/bzip2/Makefile-libbz2_so
View file

@ -1,44 +0,0 @@
# This Makefile builds a shared version of the library,
# libbz2.so.1.0.3, with soname libbz2.so.1.0,
# at least on x86-Linux (RedHat 7.2),
# with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98).
# Please see the README file for some
# important info about building the library like this.
SHELL=/bin/sh
CC=gcc
BIGFILES=-D_FILE_OFFSET_BITS=64
CFLAGS=-fpic -fPIC -Wall -Winline -O -g
OBJS= blocksort.o \
huffman.o \
crctable.o \
randtable.o \
compress.o \
decompress.o \
bzlib.o
all: $(OBJS)
$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.3 $(OBJS)
$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.3
rm -f libbz2.so.1.0
ln -s libbz2.so.1.0.3 libbz2.so.1.0
clean:
rm -f $(OBJS) bzip2.o libbz2.so.1.0.3 libbz2.so.1.0 bzip2-shared
blocksort.o: blocksort.c
$(CC) $(CFLAGS) -c blocksort.c
huffman.o: huffman.c
$(CC) $(CFLAGS) -c huffman.c
crctable.o: crctable.c
$(CC) $(CFLAGS) -c crctable.c
randtable.o: randtable.c
$(CC) $(CFLAGS) -c randtable.c
compress.o: compress.c
$(CC) $(CFLAGS) -c compress.c
decompress.o: decompress.c
$(CC) $(CFLAGS) -c decompress.c
bzlib.o: bzlib.c
$(CC) $(CFLAGS) -c bzlib.c

185
contrib/vmap_extractor_v2/stormlib/bzip2/README
View file

@ -1,185 +0,0 @@
This is the README for bzip2, a block-sorting file compressor, version
1.0.3. This version is fully compatible with the previous public
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and 1.0.2.
bzip2-1.0.3 is distributed under a BSD-style license. For details,
see the file LICENSE.
Complete documentation is available in Postscript form (manual.ps),
PDF (manual.pdf) or html (manual.html). A plain-text version of the
manual page is available as bzip2.txt. A statement about Y2K issues
is now included in the file Y2K_INFO.
HOW TO BUILD -- UNIX
Type `make'. This builds the library libbz2.a and then the
programs bzip2 and bzip2recover. Six self-tests are run.
If the self-tests complete ok, carry on to installation:
To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type
make install
To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type
make install PREFIX=/xxx/yyy
If you are (justifiably) paranoid and want to see what 'make install'
is going to do, you can first do
make -n install or
make -n install PREFIX=/xxx/yyy respectively.
The -n instructs make to show the commands it would execute, but
not actually execute them.
HOW TO BUILD -- UNIX, shared library libbz2.so.
Do 'make -f Makefile-libbz2_so'. This Makefile seems to work for
Linux-ELF (RedHat 7.2 on an x86 box), with gcc. I make no claims
that it works for any other platform, though I suspect it probably
will work for most platforms employing both ELF and gcc.
bzip2-shared, a client of the shared library, is also built, but not
self-tested. So I suggest you also build using the normal Makefile,
since that conducts a self-test. A second reason to prefer the
version statically linked to the library is that, on x86 platforms,
building shared objects makes a valuable register (%ebx) unavailable
to gcc, resulting in a slowdown of 10%-20%, at least for bzip2.
Important note for people upgrading .so's from 0.9.0/0.9.5 to version
1.0.X. All the functions in the library have been renamed, from (eg)
bzCompress to BZ2_bzCompress, to avoid namespace pollution.
Unfortunately this means that the libbz2.so created by
Makefile-libbz2_so will not work with any program which used an older
version of the library. Sorry. I do encourage library clients to
make the effort to upgrade to use version 1.0, since it is both faster
and more robust than previous versions.
HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc.
It's difficult for me to support compilation on all these platforms.
My approach is to collect binaries for these platforms, and put them
on the master web page (http://sources.redhat.com/bzip2). Look there.
However (FWIW), bzip2-1.0.X is very standard ANSI C and should compile
unmodified with MS Visual C. If you have difficulties building, you
might want to read README.COMPILATION.PROBLEMS.
At least using MS Visual C++ 6, you can build from the unmodified
sources by issuing, in a command shell:
nmake -f makefile.msc
(you may need to first run the MSVC-provided script VCVARS32.BAT
so as to set up paths to the MSVC tools correctly).
VALIDATION
Correct operation, in the sense that a compressed file can always be
decompressed to reproduce the original, is obviously of paramount
importance. To validate bzip2, I used a modified version of Mark
Nelson's churn program. Churn is an automated test driver which
recursively traverses a directory structure, using bzip2 to compress
and then decompress each file it encounters, and checking that the
decompressed data is the same as the original.
Please read and be aware of the following:
WARNING:
This program (attempts to) compress data by performing several
non-trivial transformations on it. Unless you are 100% familiar
with *all* the algorithms contained herein, and with the
consequences of modifying them, you should NOT meddle with the
compression or decompression machinery. Incorrect changes can and
very likely *will* lead to disastrous loss of data.
DISCLAIMER:
I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE
USE OF THIS PROGRAM, HOWSOEVER CAUSED.
Every compression of a file implies an assumption that the
compressed file can be decompressed to reproduce the original.
Great efforts in design, coding and testing have been made to
ensure that this program works correctly. However, the complexity
of the algorithms, and, in particular, the presence of various
special cases in the code which occur with very low but non-zero
probability make it impossible to rule out the possibility of bugs
remaining in the program. DO NOT COMPRESS ANY DATA WITH THIS
PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE POSSIBILITY, HOWEVER
SMALL, THAT THE DATA WILL NOT BE RECOVERABLE.
That is not to say this program is inherently unreliable. Indeed,
I very much hope the opposite is true. bzip2 has been carefully
constructed and extensively tested.
PATENTS:
To the best of my knowledge, bzip2 does not use any patented
algorithms. However, I do not have the resources to carry out
a patent search. Therefore I cannot give any guarantee of the
above statement.
End of legalities.
WHAT'S NEW IN 0.9.0 (as compared to 0.1pl2) ?
* Approx 10% faster compression, 30% faster decompression
* -t (test mode) is a lot quicker
* Can decompress concatenated compressed files
* Programming interface, so programs can directly read/write .bz2 files
* Less restrictive (BSD-style) licensing
* Flag handling more compatible with GNU gzip
* Much more documentation, i.e., a proper user manual
* Hopefully, improved portability (at least of the library)
WHAT'S NEW IN 0.9.5 ?
* Compression speed is much less sensitive to the input
data than in previous versions. Specifically, the very
slow performance caused by repetitive data is fixed.
* Many small improvements in file and flag handling.
* A Y2K statement.
WHAT'S NEW IN 1.0.0 ?
See the CHANGES file.
WHAT'S NEW IN 1.0.2 ?
See the CHANGES file.
WHAT'S NEW IN 1.0.3 ?
See the CHANGES file.
I hope you find bzip2 useful. Feel free to contact me at
jseward@bzip.org
if you have any suggestions or queries. Many people mailed me with
comments, suggestions and patches after the releases of bzip-0.15,
bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
1.0.2, and the changes in bzip2 are largely a result of this feedback.
I thank you for your comments.
At least for the time being, bzip2's "home" is (or can be reached via)
http://www.bzip.org
Julian Seward
jseward@bzip.org
Cambridge, UK.
18 July 1996 (version 0.15)
25 August 1996 (version 0.21)
7 August 1997 (bzip2, version 0.1)
29 August 1997 (bzip2, version 0.1pl2)
23 August 1998 (bzip2, version 0.9.0)
8 June 1999 (bzip2, version 0.9.5)
4 Sept 1999 (bzip2, version 0.9.5d)
5 May 2000 (bzip2, version 1.0pre8)
30 December 2001 (bzip2, version 1.0.2pre1)
15 February 2005 (bzip2, version 1.0.3)

39
contrib/vmap_extractor_v2/stormlib/bzip2/README.COMPILATION.PROBLEMS
View file

@ -1,39 +0,0 @@
bzip2-1.0.3 should compile without problems on the vast majority of
platforms. Using the supplied Makefile, I've built and tested it
myself for x86-linux and x86_64-linux. With makefile.msc, Visual C++
6.0 and nmake, you can build a native Win32 version too. Large file
support seems to work correctly on at least alpha-tru64unix and
x86-cygwin32 (on Windows 2000).
When I say "large file" I mean a file of size 2,147,483,648 (2^31)
bytes or above. Many older OSs can't handle files above this size,
but many newer ones can. Large files are pretty huge -- most files
you'll encounter are not Large Files.
Earlier versions of bzip2 (0.1, 0.9.0, 0.9.5) compiled on a wide
variety of platforms without difficulty, and I hope this version will
continue in that tradition. However, in order to support large files,
I've had to include the define -D_FILE_OFFSET_BITS=64 in the Makefile.
This can cause problems.
The technique of adding -D_FILE_OFFSET_BITS=64 to get large file
support is, as far as I know, the Recommended Way to get correct large
file support. For more details, see the Large File Support
Specification, published by the Large File Summit, at
http://ftp.sas.com/standards/large.file
As a general comment, if you get compilation errors which you think
are related to large file support, try removing the above define from
the Makefile, ie, delete the line
BIGFILES=-D_FILE_OFFSET_BITS=64
from the Makefile, and do 'make clean ; make'. This will give you a
version of bzip2 without large file support, which, for most
applications, is probably not a problem.
Alternatively, try some of the platform-specific hints listed below.
You can use the spewG.c program to generate huge files to test bzip2's
large file support, if you are feeling paranoid. Be aware though that
any compilation problems which affect bzip2 will also affect spewG.c,
alas.

31
contrib/vmap_extractor_v2/stormlib/bzip2/README.XML.STUFF
View file

@ -1,31 +0,0 @@
The script xmlproc.sh takes an xml file as input,
and processes it to create .pdf, .html or .ps output.
It uses format.pl, a perl script to format <pre> blocks nicely,
and add CDATA tags so writers do not have to use eg. &lt;
The file "entities.xml" must be edited to reflect current
version, year, etc.
Usage:
xmlproc.sh -v manual.xml
Validates an xml file to ensure no dtd-compliance errors
xmlproc.sh -html manual.xml
Output: manual.html
xmlproc.sh -pdf manual.xml
Output: manual.pdf
xmlproc.sh -ps manual.xml
Output: manual.ps
Notum bene:
- pdfxmltex barfs if given a filename with an underscore in it
- xmltex won't work yet - there's a bug in passivetex
which we are all waiting for Sebastian to fix.
So we are going the xml -> pdf -> ps route for the time being,
using pdfxmltex.

34
contrib/vmap_extractor_v2/stormlib/bzip2/Y2K_INFO
View file

@ -1,34 +0,0 @@
Y2K status of bzip2 and libbzip2, versions 0.1, 0.9.0 and 0.9.5
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Informally speaking:
bzip2 is a compression program built on top of libbzip2,
a library which does the real work of compression and
decompression. As far as I am aware, libbzip2 does not have
any date-related code at all.
bzip2 itself copies dates from source to destination files
when compressing or decompressing, using the 'stat' and 'utime'
UNIX system calls. It doesn't examine, manipulate or store the
dates in any way. So as far as I can see, there shouldn't be any
problem with bzip2 providing 'stat' and 'utime' work correctly
on your system.
On non-unix platforms (those for which BZ_UNIX in bzip2.c is
not set to 1), bzip2 doesn't even do the date copying.
Overall, informally speaking, I don't think bzip2 or libbzip2
have a Y2K problem.
Formally speaking:
I am not prepared to offer you any assurance whatsoever
regarding Y2K issues in my software. You alone assume the
entire risk of using the software. The disclaimer of liability
in the LICENSE file in the bzip2 source distribution continues
to apply on this issue as with every other issue pertaining
to the software.
Julian Seward
Cambridge, UK
25 August 1999

1141
contrib/vmap_extractor_v2/stormlib/bzip2/blocksort.c
View file

File diff suppressed because it is too large Load diff

39
contrib/vmap_extractor_v2/stormlib/bzip2/bz-common.xsl
View file

@ -1,39 +0,0 @@
<?xml version="1.0"?> <!-- -*- sgml -*- -->
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<!-- we like '1.2 Title' -->
<xsl:param name="section.autolabel" select="'1'"/>
<xsl:param name="section.label.includes.component.label" select="'1'"/>
<!-- Do not put 'Chapter' at the start of eg 'Chapter 1. Doing This' -->
<xsl:param name="local.l10n.xml" select="document('')"/>
<l:i18n xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">
<l:l10n language="en">
<l:context name="title-numbered">
<l:template name="chapter" text="%n.&#160;%t"/>
</l:context>
</l:l10n>
</l:i18n>
<!-- don't generate sub-tocs for qanda sets -->
<xsl:param name="generate.toc">
set toc,title
book toc,title,figure,table,example,equation
chapter toc,title
section toc
sect1 toc
sect2 toc
sect3 toc
sect4 nop
sect5 nop
qandaset toc
qandadiv nop
appendix toc,title
article/appendix nop
article toc,title
preface toc,title
reference toc,title
</xsl:param>
</xsl:stylesheet>

257
contrib/vmap_extractor_v2/stormlib/bzip2/bz-fo.xsl
View file

@ -1,257 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?> <!-- -*- sgml -*- -->
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
<xsl:import href="http://docbook.sourceforge.net/release/xsl/current/fo/docbook.xsl"/>
<xsl:import href="bz-common.xsl"/>
<!-- set indent = yes while debugging, then change to NO -->
<xsl:output method="xml" indent="yes"/>
<!-- ensure only passivetex extensions are on -->
<xsl:param name="stylesheet.result.type" select="'fo'"/>
<!-- fo extensions: PDF bookmarks and index terms -->
<xsl:param name="use.extensions" select="'1'"/>
<xsl:param name="xep.extensions" select="0"/>
<xsl:param name="fop.extensions" select="0"/>
<xsl:param name="saxon.extensions" select="0"/>
<xsl:param name="passivetex.extensions" select="1"/>
<xsl:param name="tablecolumns.extension" select="'1'"/>
<!-- ensure we are using single sided -->
<xsl:param name="double.sided" select="'0'"/>
<!-- insert cross references to page numbers -->
<xsl:param name="insert.xref.page.number" select="1"/>
<!-- <?custom-pagebreak?> inserts a page break at this point -->
<xsl:template match="processing-instruction('custom-pagebreak')">
<fo:block break-before='page'/>
</xsl:template>
<!-- show links in color -->
<xsl:attribute-set name="xref.properties">
<xsl:attribute name="color">blue</xsl:attribute>
</xsl:attribute-set>
<!-- make pre listings indented a bit + a bg colour -->
<xsl:template match="programlisting | screen">
<fo:block start-indent="0.25in" wrap-option="no-wrap"
white-space-collapse="false" text-align="start"
font-family="monospace" background-color="#f2f2f9"
linefeed-treatment="preserve"
xsl:use-attribute-sets="normal.para.spacing">
<xsl:apply-templates/>
</fo:block>
</xsl:template>
<!-- make verbatim output prettier -->
<xsl:template match="literallayout">
<fo:block start-indent="0.25in" wrap-option="no-wrap"
white-space-collapse="false" text-align="start"
font-family="monospace" background-color="#edf7f4"
linefeed-treatment="preserve"
space-before="0em" space-after="0em">
<xsl:apply-templates/>
</fo:block>
</xsl:template>
<!-- workaround bug in passivetex fo output for itemizedlist -->
<xsl:template match="itemizedlist/listitem">
<xsl:variable name="id">
<xsl:call-template name="object.id"/></xsl:variable>
<xsl:variable name="itemsymbol">
<xsl:call-template name="list.itemsymbol">
<xsl:with-param name="node" select="parent::itemizedlist"/>
</xsl:call-template>
</xsl:variable>
<xsl:variable name="item.contents">
<fo:list-item-label end-indent="label-end()">
<fo:block>
<xsl:choose>
<xsl:when test="$itemsymbol='disc'">&#x2022;</xsl:when>
<xsl:when test="$itemsymbol='bullet'">&#x2022;</xsl:when>
<xsl:otherwise>&#x2022;</xsl:otherwise>
</xsl:choose>
</fo:block>
</fo:list-item-label>
<fo:list-item-body start-indent="body-start()">
<xsl:apply-templates/> <!-- removed extra block wrapper -->
</fo:list-item-body>
</xsl:variable>
<xsl:choose>
<xsl:when test="parent::*/@spacing = 'compact'">
<fo:list-item id="{$id}"
xsl:use-attribute-sets="compact.list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:when>
<xsl:otherwise>
<fo:list-item id="{$id}" xsl:use-attribute-sets="list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- workaround bug in passivetex fo output for orderedlist -->
<xsl:template match="orderedlist/listitem">
<xsl:variable name="id">
<xsl:call-template name="object.id"/></xsl:variable>
<xsl:variable name="item.contents">
<fo:list-item-label end-indent="label-end()">
<fo:block>
<xsl:apply-templates select="." mode="item-number"/>
</fo:block>
</fo:list-item-label>
<fo:list-item-body start-indent="body-start()">
<xsl:apply-templates/> <!-- removed extra block wrapper -->
</fo:list-item-body>
</xsl:variable>
<xsl:choose>
<xsl:when test="parent::*/@spacing = 'compact'">
<fo:list-item id="{$id}"
xsl:use-attribute-sets="compact.list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:when>
<xsl:otherwise>
<fo:list-item id="{$id}" xsl:use-attribute-sets="list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- workaround bug in passivetex fo output for variablelist -->
<xsl:param name="variablelist.as.blocks" select="1"/>
<xsl:template match="varlistentry" mode="vl.as.blocks">
<xsl:variable name="id">
<xsl:call-template name="object.id"/></xsl:variable>
<fo:block id="{$id}" xsl:use-attribute-sets="list.item.spacing"
keep-together.within-column="always"
keep-with-next.within-column="always">
<xsl:apply-templates select="term"/>
</fo:block>
<fo:block start-indent="0.5in" end-indent="0in"
space-after.minimum="0.2em"
space-after.optimum="0.4em"
space-after.maximum="0.6em">
<fo:block>
<xsl:apply-templates select="listitem"/>
</fo:block>
</fo:block>
</xsl:template>
<!-- workaround bug in footers: force right-align w/two 80|30 cols -->
<xsl:template name="footer.table">
<xsl:param name="pageclass" select="''"/>
<xsl:param name="sequence" select="''"/>
<xsl:param name="gentext-key" select="''"/>
<xsl:choose>
<xsl:when test="$pageclass = 'index'">
<xsl:attribute name="margin-left">0pt</xsl:attribute>
</xsl:when>
</xsl:choose>
<xsl:variable name="candidate">
<fo:table table-layout="fixed" width="100%">
<fo:table-column column-number="1" column-width="80%"/>
<fo:table-column column-number="2" column-width="20%"/>
<fo:table-body>
<fo:table-row height="14pt">
<fo:table-cell text-align="left" display-align="after">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<fo:block> </fo:block><!-- empty cell -->
</fo:block>
</fo:table-cell>
<fo:table-cell text-align="center" display-align="after">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<xsl:call-template name="footer.content">
<xsl:with-param name="pageclass" select="$pageclass"/>
<xsl:with-param name="sequence" select="$sequence"/>
<xsl:with-param name="position" select="'center'"/>
<xsl:with-param name="gentext-key" select="$gentext-key"/>
</xsl:call-template>
</fo:block>
</fo:table-cell>
</fo:table-row>
</fo:table-body>
</fo:table>
</xsl:variable>
<!-- Really output a footer? -->
<xsl:choose>
<xsl:when test="$pageclass='titlepage' and $gentext-key='book'
and $sequence='first'">
<!-- no, book titlepages have no footers at all -->
</xsl:when>
<xsl:when test="$sequence = 'blank' and $footers.on.blank.pages = 0">
<!-- no output -->
</xsl:when>
<xsl:otherwise>
<xsl:copy-of select="$candidate"/>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- fix bug in headers: force right-align w/two 40|60 cols -->
<xsl:template name="header.table">
<xsl:param name="pageclass" select="''"/>
<xsl:param name="sequence" select="''"/>
<xsl:param name="gentext-key" select="''"/>
<xsl:choose>
<xsl:when test="$pageclass = 'index'">
<xsl:attribute name="margin-left">0pt</xsl:attribute>
</xsl:when>
</xsl:choose>
<xsl:variable name="candidate">
<fo:table table-layout="fixed" width="100%">
<xsl:call-template name="head.sep.rule">
<xsl:with-param name="pageclass" select="$pageclass"/>
<xsl:with-param name="sequence" select="$sequence"/>
<xsl:with-param name="gentext-key" select="$gentext-key"/>
</xsl:call-template>
<fo:table-column column-number="1" column-width="40%"/>
<fo:table-column column-number="2" column-width="60%"/>
<fo:table-body>
<fo:table-row height="14pt">
<fo:table-cell text-align="left" display-align="before">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<fo:block> </fo:block><!-- empty cell -->
</fo:block>
</fo:table-cell>
<fo:table-cell text-align="center" display-align="before">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<xsl:call-template name="header.content">
<xsl:with-param name="pageclass" select="$pageclass"/>
<xsl:with-param name="sequence" select="$sequence"/>
<xsl:with-param name="position" select="'center'"/>
<xsl:with-param name="gentext-key" select="$gentext-key"/>
</xsl:call-template>
</fo:block>
</fo:table-cell>
</fo:table-row>
</fo:table-body>
</fo:table>
</xsl:variable>
<!-- Really output a header? -->
<xsl:choose>
<xsl:when test="$pageclass = 'titlepage' and $gentext-key = 'book'
and $sequence='first'">
<!-- no, book titlepages have no headers at all -->
</xsl:when>
<xsl:when test="$sequence = 'blank' and $headers.on.blank.pages = 0">
<!-- no output -->
</xsl:when>
<xsl:otherwise>
<xsl:copy-of select="$candidate"/>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
</xsl:stylesheet>

20
contrib/vmap_extractor_v2/stormlib/bzip2/bz-html.xsl
View file

@ -1,20 +0,0 @@
<?xml version="1.0"?> <!-- -*- sgml -*- -->
<!DOCTYPE xsl:stylesheet [ <!ENTITY bz-css SYSTEM "./bzip.css"> ]>
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:import href="http://docbook.sourceforge.net/release/xsl/current/html/docbook.xsl"/>
<xsl:import href="bz-common.xsl"/>
<!-- use 8859-1 encoding -->
<xsl:output method="html" encoding="ISO-8859-1" indent="yes"/>
<!-- we include the css directly when generating one large file -->
<xsl:template name="user.head.content">
<style type="text/css" media="screen">
<xsl:text>&bz-css;</xsl:text>
</style>
</xsl:template>
</xsl:stylesheet>

76
contrib/vmap_extractor_v2/stormlib/bzip2/bzdiff
View file

@ -1,76 +0,0 @@
#!/bin/sh
# sh is buggy on RS/6000 AIX 3.2. Replace above line with #!/bin/ksh
# Bzcmp/diff wrapped for bzip2,
# adapted from zdiff by Philippe Troin <phil@fifi.org> for Debian GNU/Linux.
# Bzcmp and bzdiff are used to invoke the cmp or the diff pro-
# gram on compressed files. All options specified are passed
# directly to cmp or diff. If only 1 file is specified, then
# the files compared are file1 and an uncompressed file1.gz.
# If two files are specified, then they are uncompressed (if
# necessary) and fed to cmp or diff. The exit status from cmp
# or diff is preserved.
PATH="/usr/bin:$PATH"; export PATH
prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
*cmp) comp=${CMP-cmp} ;;
*) comp=${DIFF-diff} ;;
esac
OPTIONS=
FILES=
for ARG
do
case "$ARG" in
-*) OPTIONS="$OPTIONS $ARG";;
*) if test -f "$ARG"; then
FILES="$FILES $ARG"
else
echo "${prog}: $ARG not found or not a regular file"
exit 1
fi ;;
esac
done
if test -z "$FILES"; then
echo "Usage: $prog [${comp}_options] file [file]"
exit 1
fi
tmp=`tempfile -d /tmp -p bz` || {
echo 'cannot create a temporary file' >&2
exit 1
}
set $FILES
if test $# -eq 1; then
FILE=`echo "$1" | sed 's/.bz2$//'`
bzip2 -cd "$FILE.bz2" | $comp $OPTIONS - "$FILE"
STAT="$?"
elif test $# -eq 2; then
case "$1" in
*.bz2)
case "$2" in
*.bz2)
F=`echo "$2" | sed 's|.*/||;s|.bz2$||'`
bzip2 -cdfq "$2" > $tmp
bzip2 -cdfq "$1" | $comp $OPTIONS - $tmp
STAT="$?"
/bin/rm -f $tmp;;
*) bzip2 -cdfq "$1" | $comp $OPTIONS - "$2"
STAT="$?";;
esac;;
*) case "$2" in
*.bz2)
bzip2 -cdfq "$2" | $comp $OPTIONS "$1" -
STAT="$?";;
*) $comp $OPTIONS "$1" "$2"
STAT="$?";;
esac;;
esac
exit "$STAT"
else
echo "Usage: $prog [${comp}_options] file [file]"
exit 1
fi

47
contrib/vmap_extractor_v2/stormlib/bzip2/bzdiff.1
View file

@ -1,47 +0,0 @@
\"Shamelessly copied from zmore.1 by Philippe Troin <phil@fifi.org>
\"for Debian GNU/Linux
.TH BZDIFF 1
.SH NAME
bzcmp, bzdiff \- compare bzip2 compressed files
.SH SYNOPSIS
.B bzcmp
[ cmp_options ] file1
[ file2 ]
.br
.B bzdiff
[ diff_options ] file1
[ file2 ]
.SH DESCRIPTION
.I Bzcmp
and
.I bzdiff
are used to invoke the
.I cmp
or the
.I diff
program on bzip2 compressed files. All options specified are passed
directly to
.I cmp
or
.IR diff "."
If only 1 file is specified, then the files compared are
.I file1
and an uncompressed
.IR file1 ".bz2."
If two files are specified, then they are uncompressed if necessary and fed to
.I cmp
or
.IR diff "."
The exit status from
.I cmp
or
.I diff
is preserved.
.SH "SEE ALSO"
cmp(1), diff(1), bzmore(1), bzless(1), bzgrep(1), bzip2(1)
.SH BUGS
Messages from the
.I cmp
or
.I diff
programs refer to temporary filenames instead of those specified.

71
contrib/vmap_extractor_v2/stormlib/bzip2/bzgrep
View file

@ -1,71 +0,0 @@
#!/bin/sh
# Bzgrep wrapped for bzip2,
# adapted from zgrep by Philippe Troin <phil@fifi.org> for Debian GNU/Linux.
## zgrep notice:
## zgrep -- a wrapper around a grep program that decompresses files as needed
## Adapted from a version sent by Charles Levert <charles@comm.polymtl.ca>
PATH="/usr/bin:$PATH"; export PATH
prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
*egrep) grep=${EGREP-egrep} ;;
*fgrep) grep=${FGREP-fgrep} ;;
*) grep=${GREP-grep} ;;
esac
pat=""
while test $# -ne 0; do
case "$1" in
-e | -f) opt="$opt $1"; shift; pat="$1"
if test "$grep" = grep; then # grep is buggy with -e on SVR4
grep=egrep
fi;;
-A | -B) opt="$opt $1 $2"; shift;;
-*) opt="$opt $1";;
*) if test -z "$pat"; then
pat="$1"
else
break;
fi;;
esac
shift
done
if test -z "$pat"; then
echo "grep through bzip2 files"
echo "usage: $prog [grep_options] pattern [files]"
exit 1
fi
list=0
silent=0
op=`echo "$opt" | sed -e 's/ //g' -e 's/-//g'`
case "$op" in
*l*) list=1
esac
case "$op" in
*h*) silent=1
esac
if test $# -eq 0; then
bzip2 -cdfq | $grep $opt "$pat"
exit $?
fi
res=0
for i do
if test -f "$i"; then :; else if test -f "$i.bz2"; then i="$i.bz2"; fi; fi
if test $list -eq 1; then
bzip2 -cdfq "$i" | $grep $opt "$pat" 2>&1 > /dev/null && echo $i
r=$?
elif test $# -eq 1 -o $silent -eq 1; then
bzip2 -cdfq "$i" | $grep $opt "$pat"
r=$?
else
bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${i}:|"
r=$?
fi
test "$r" -ne 0 && res="$r"
done
exit $res

56
contrib/vmap_extractor_v2/stormlib/bzip2/bzgrep.1
View file

@ -1,56 +0,0 @@
\"Shamelessly copied from zmore.1 by Philippe Troin <phil@fifi.org>
\"for Debian GNU/Linux
.TH BZGREP 1
.SH NAME
bzgrep, bzfgrep, bzegrep \- search possibly bzip2 compressed files for a regular expression
.SH SYNOPSIS
.B bzgrep
[ grep_options ]
.BI [\ -e\ ] " pattern"
.IR filename ".\|.\|."
.br
.B bzegrep
[ egrep_options ]
.BI [\ -e\ ] " pattern"
.IR filename ".\|.\|."
.br
.B bzfgrep
[ fgrep_options ]
.BI [\ -e\ ] " pattern"
.IR filename ".\|.\|."
.SH DESCRIPTION
.IR Bzgrep
is used to invoke the
.I grep
on bzip2-compressed files. All options specified are passed directly to
.I grep.
If no file is specified, then the standard input is decompressed
if necessary and fed to grep.
Otherwise the given files are uncompressed if necessary and fed to
.I grep.
.PP
If
.I bzgrep
is invoked as
.I bzegrep
or
.I bzfgrep
then
.I egrep
or
.I fgrep
is used instead of
.I grep.
If the GREP environment variable is set,
.I bzgrep
uses it as the
.I grep
program to be invoked. For example:
for sh: GREP=fgrep bzgrep string files
for csh: (setenv GREP fgrep; bzgrep string files)
.SH AUTHOR
Charles Levert (charles@comm.polymtl.ca). Adapted to bzip2 by Philippe
Troin <phil@fifi.org> for Debian GNU/Linux.
.SH "SEE ALSO"
grep(1), egrep(1), fgrep(1), bzdiff(1), bzmore(1), bzless(1), bzip2(1)

74
contrib/vmap_extractor_v2/stormlib/bzip2/bzip.css
View file

@ -1,74 +0,0 @@
/* Colours:
#74240f dark brown h1, h2, h3, h4
#336699 medium blue links
#339999 turquoise link hover colour
#202020 almost black general text
#761596 purple md5sum text
#626262 dark gray pre border
#eeeeee very light gray pre background
#f2f2f9 very light blue nav table background
#3366cc medium blue nav table border
*/
a, a:link, a:visited, a:active { color: #336699; }
a:hover { color: #339999; }
body { font: 80%/126% sans-serif; }
h1, h2, h3, h4 { color: #74240f; }
dt { color: #336699; font-weight: bold }
dd {
margin-left: 1.5em;
padding-bottom: 0.8em;
}
/* -- ruler -- */
div.hr_blue {
height: 3px;
background:#ffffff url("/images/hr_blue.png") repeat-x; }
div.hr_blue hr { display:none; }
/* release styles */
#release p { margin-top: 0.4em; }
#release .md5sum { color: #761596; }
/* ------ styles for docs|manuals|howto ------ */
/* -- lists -- */
ul {
margin: 0px 4px 16px 16px;
padding: 0px;
list-style: url("/images/li-blue.png");
}
ul li {
margin-bottom: 10px;
}
ul ul {
list-style-type: none;
list-style-image: none;
margin-left: 0px;
}
/* header / footer nav tables */
table.nav {
border: solid 1px #3366cc;
background: #f2f2f9;
background-color: #f2f2f9;
margin-bottom: 0.5em;
}
/* don't have underlined links in chunked nav menus */
table.nav a { text-decoration: none; }
table.nav a:hover { text-decoration: underline; }
table.nav td { font-size: 85%; }
code, tt, pre { font-size: 120%; }
code, tt { color: #761596; }
div.literallayout, pre.programlisting, pre.screen {
color: #000000;
padding: 0.5em;
background: #eeeeee;
border: 1px solid #626262;
background-color: #eeeeee;
margin: 4px 0px 4px 0px;
}

454
contrib/vmap_extractor_v2/stormlib/bzip2/bzip2.1
View file

@ -1,454 +0,0 @@
.PU
.TH bzip2 1
.SH NAME
bzip2, bunzip2 \- a block-sorting file compressor, v1.0.3
.br
bzcat \- decompresses files to stdout
.br
bzip2recover \- recovers data from damaged bzip2 files
.SH SYNOPSIS
.ll +8
.B bzip2
.RB [ " \-cdfkqstvzVL123456789 " ]
[
.I "filenames \&..."
]
.ll -8
.br
.B bunzip2
.RB [ " \-fkvsVL " ]
[
.I "filenames \&..."
]
.br
.B bzcat
.RB [ " \-s " ]
[
.I "filenames \&..."
]
.br
.B bzip2recover
.I "filename"
.SH DESCRIPTION
.I bzip2
compresses files using the Burrows-Wheeler block sorting
text compression algorithm, and Huffman coding. Compression is
generally considerably better than that achieved by more conventional
LZ77/LZ78-based compressors, and approaches the performance of the PPM
family of statistical compressors.
The command-line options are deliberately very similar to
those of
.I GNU gzip,
but they are not identical.
.I bzip2
expects a list of file names to accompany the
command-line flags. Each file is replaced by a compressed version of
itself, with the name "original_name.bz2".
Each compressed file
has the same modification date, permissions, and, when possible,
ownership as the corresponding original, so that these properties can
be correctly restored at decompression time. File name handling is
naive in the sense that there is no mechanism for preserving original
file names, permissions, ownerships or dates in filesystems which lack
these concepts, or have serious file name length restrictions, such as
MS-DOS.
.I bzip2
and
.I bunzip2
will by default not overwrite existing
files. If you want this to happen, specify the \-f flag.
If no file names are specified,
.I bzip2
compresses from standard
input to standard output. In this case,
.I bzip2
will decline to
write compressed output to a terminal, as this would be entirely
incomprehensible and therefore pointless.
.I bunzip2
(or
.I bzip2 \-d)
decompresses all
specified files. Files which were not created by
.I bzip2
will be detected and ignored, and a warning issued.
.I bzip2
attempts to guess the filename for the decompressed file
from that of the compressed file as follows:
filename.bz2 becomes filename
filename.bz becomes filename
filename.tbz2 becomes filename.tar
filename.tbz becomes filename.tar
anyothername becomes anyothername.out
If the file does not end in one of the recognised endings,
.I .bz2,
.I .bz,
.I .tbz2
or
.I .tbz,
.I bzip2
complains that it cannot
guess the name of the original file, and uses the original name
with
.I .out
appended.
As with compression, supplying no
filenames causes decompression from
standard input to standard output.
.I bunzip2
will correctly decompress a file which is the
concatenation of two or more compressed files. The result is the
concatenation of the corresponding uncompressed files. Integrity
testing (\-t)
of concatenated
compressed files is also supported.
You can also compress or decompress files to the standard output by
giving the \-c flag. Multiple files may be compressed and
decompressed like this. The resulting outputs are fed sequentially to
stdout. Compression of multiple files
in this manner generates a stream
containing multiple compressed file representations. Such a stream
can be decompressed correctly only by
.I bzip2
version 0.9.0 or
later. Earlier versions of
.I bzip2
will stop after decompressing
the first file in the stream.
.I bzcat
(or
.I bzip2 -dc)
decompresses all specified files to
the standard output.
.I bzip2
will read arguments from the environment variables
.I BZIP2
and
.I BZIP,
in that order, and will process them
before any arguments read from the command line. This gives a
convenient way to supply default arguments.
Compression is always performed, even if the compressed
file is slightly
larger than the original. Files of less than about one hundred bytes
tend to get larger, since the compression mechanism has a constant
overhead in the region of 50 bytes. Random data (including the output
of most file compressors) is coded at about 8.05 bits per byte, giving
an expansion of around 0.5%.
As a self-check for your protection,
.I
bzip2
uses 32-bit CRCs to
make sure that the decompressed version of a file is identical to the
original. This guards against corruption of the compressed data, and
against undetected bugs in
.I bzip2
(hopefully very unlikely). The
chances of data corruption going undetected is microscopic, about one
chance in four billion for each file processed. Be aware, though, that
the check occurs upon decompression, so it can only tell you that
something is wrong. It can't help you
recover the original uncompressed
data. You can use
.I bzip2recover
to try to recover data from
damaged files.
Return values: 0 for a normal exit, 1 for environmental problems (file
not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
compressed file, 3 for an internal consistency error (eg, bug) which
caused
.I bzip2
to panic.
.SH OPTIONS
.TP
.B \-c --stdout
Compress or decompress to standard output.
.TP
.B \-d --decompress
Force decompression.
.I bzip2,
.I bunzip2
and
.I bzcat
are
really the same program, and the decision about what actions to take is
done on the basis of which name is used. This flag overrides that
mechanism, and forces
.I bzip2
to decompress.
.TP
.B \-z --compress
The complement to \-d: forces compression, regardless of the
invocation name.
.TP
.B \-t --test
Check integrity of the specified file(s), but don't decompress them.
This really performs a trial decompression and throws away the result.
.TP
.B \-f --force
Force overwrite of output files. Normally,
.I bzip2
will not overwrite
existing output files. Also forces
.I bzip2
to break hard links
to files, which it otherwise wouldn't do.
bzip2 normally declines to decompress files which don't have the
correct magic header bytes. If forced (-f), however, it will pass
such files through unmodified. This is how GNU gzip behaves.
.TP
.B \-k --keep
Keep (don't delete) input files during compression
or decompression.
.TP
.B \-s --small
Reduce memory usage, for compression, decompression and testing. Files
are decompressed and tested using a modified algorithm which only
requires 2.5 bytes per block byte. This means any file can be
decompressed in 2300k of memory, albeit at about half the normal speed.
During compression, \-s selects a block size of 200k, which limits
memory use to around the same figure, at the expense of your compression
ratio. In short, if your machine is low on memory (8 megabytes or
less), use \-s for everything. See MEMORY MANAGEMENT below.
.TP
.B \-q --quiet
Suppress non-essential warning messages. Messages pertaining to
I/O errors and other critical events will not be suppressed.
.TP
.B \-v --verbose
Verbose mode -- show the compression ratio for each file processed.
Further \-v's increase the verbosity level, spewing out lots of
information which is primarily of interest for diagnostic purposes.
.TP
.B \-L --license -V --version
Display the software version, license terms and conditions.
.TP
.B \-1 (or \-\-fast) to \-9 (or \-\-best)
Set the block size to 100 k, 200 k .. 900 k when compressing. Has no
effect when decompressing. See MEMORY MANAGEMENT below.
The \-\-fast and \-\-best aliases are primarily for GNU gzip
compatibility. In particular, \-\-fast doesn't make things
significantly faster.
And \-\-best merely selects the default behaviour.
.TP
.B \--
Treats all subsequent arguments as file names, even if they start
with a dash. This is so you can handle files with names beginning
with a dash, for example: bzip2 \-- \-myfilename.
.TP
.B \--repetitive-fast --repetitive-best
These flags are redundant in versions 0.9.5 and above. They provided
some coarse control over the behaviour of the sorting algorithm in
earlier versions, which was sometimes useful. 0.9.5 and above have an
improved algorithm which renders these flags irrelevant.
.SH MEMORY MANAGEMENT
.I bzip2
compresses large files in blocks. The block size affects
both the compression ratio achieved, and the amount of memory needed for
compression and decompression. The flags \-1 through \-9
specify the block size to be 100,000 bytes through 900,000 bytes (the
default) respectively. At decompression time, the block size used for
compression is read from the header of the compressed file, and
.I bunzip2
then allocates itself just enough memory to decompress
the file. Since block sizes are stored in compressed files, it follows
that the flags \-1 to \-9 are irrelevant to and so ignored
during decompression.
Compression and decompression requirements,
in bytes, can be estimated as:
Compression: 400k + ( 8 x block size )
Decompression: 100k + ( 4 x block size ), or
100k + ( 2.5 x block size )
Larger block sizes give rapidly diminishing marginal returns. Most of
the compression comes from the first two or three hundred k of block
size, a fact worth bearing in mind when using
.I bzip2
on small machines.
It is also important to appreciate that the decompression memory
requirement is set at compression time by the choice of block size.
For files compressed with the default 900k block size,
.I bunzip2
will require about 3700 kbytes to decompress. To support decompression
of any file on a 4 megabyte machine,
.I bunzip2
has an option to
decompress using approximately half this amount of memory, about 2300
kbytes. Decompression speed is also halved, so you should use this
option only where necessary. The relevant flag is -s.
In general, try and use the largest block size memory constraints allow,
since that maximises the compression achieved. Compression and
decompression speed are virtually unaffected by block size.
Another significant point applies to files which fit in a single block
-- that means most files you'd encounter using a large block size. The
amount of real memory touched is proportional to the size of the file,
since the file is smaller than a block. For example, compressing a file
20,000 bytes long with the flag -9 will cause the compressor to
allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560
kbytes of it. Similarly, the decompressor will allocate 3700k but only
touch 100k + 20000 * 4 = 180 kbytes.
Here is a table which summarises the maximum memory usage for different
block sizes. Also recorded is the total compressed size for 14 files of
the Calgary Text Compression Corpus totalling 3,141,622 bytes. This
column gives some feel for how compression varies with block size.
These figures tend to understate the advantage of larger block sizes for
larger files, since the Corpus is dominated by smaller files.
Compress Decompress Decompress Corpus
Flag usage usage -s usage Size
-1 1200k 500k 350k 914704
-2 2000k 900k 600k 877703
-3 2800k 1300k 850k 860338
-4 3600k 1700k 1100k 846899
-5 4400k 2100k 1350k 845160
-6 5200k 2500k 1600k 838626
-7 6100k 2900k 1850k 834096
-8 6800k 3300k 2100k 828642
-9 7600k 3700k 2350k 828642
.SH RECOVERING DATA FROM DAMAGED FILES
.I bzip2
compresses files in blocks, usually 900kbytes long. Each
block is handled independently. If a media or transmission error causes
a multi-block .bz2
file to become damaged, it may be possible to
recover data from the undamaged blocks in the file.
The compressed representation of each block is delimited by a 48-bit
pattern, which makes it possible to find the block boundaries with
reasonable certainty. Each block also carries its own 32-bit CRC, so
damaged blocks can be distinguished from undamaged ones.
.I bzip2recover
is a simple program whose purpose is to search for
blocks in .bz2 files, and write each block out into its own .bz2
file. You can then use
.I bzip2
\-t
to test the
integrity of the resulting files, and decompress those which are
undamaged.
.I bzip2recover
takes a single argument, the name of the damaged file,
and writes a number of files "rec00001file.bz2",
"rec00002file.bz2", etc, containing the extracted blocks.
The output filenames are designed so that the use of
wildcards in subsequent processing -- for example,
"bzip2 -dc rec*file.bz2 > recovered_data" -- processes the files in
the correct order.
.I bzip2recover
should be of most use dealing with large .bz2
files, as these will contain many blocks. It is clearly
futile to use it on damaged single-block files, since a
damaged block cannot be recovered. If you wish to minimise
any potential data loss through media or transmission errors,
you might consider compressing with a smaller
block size.
.SH PERFORMANCE NOTES
The sorting phase of compression gathers together similar strings in the
file. Because of this, files containing very long runs of repeated
symbols, like "aabaabaabaab ..." (repeated several hundred times) may
compress more slowly than normal. Versions 0.9.5 and above fare much
better than previous versions in this respect. The ratio between
worst-case and average-case compression time is in the region of 10:1.
For previous versions, this figure was more like 100:1. You can use the
\-vvvv option to monitor progress in great detail, if you want.
Decompression speed is unaffected by these phenomena.
.I bzip2
usually allocates several megabytes of memory to operate
in, and then charges all over it in a fairly random fashion. This means
that performance, both for compressing and decompressing, is largely
determined by the speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the miss rate have
been observed to give disproportionately large performance improvements.
I imagine
.I bzip2
will perform best on machines with very large caches.
.SH CAVEATS
I/O error messages are not as helpful as they could be.
.I bzip2
tries hard to detect I/O errors and exit cleanly, but the details of
what the problem is sometimes seem rather misleading.
This manual page pertains to version 1.0.3 of
.I bzip2.
Compressed data created by this version is entirely forwards and
backwards compatible with the previous public releases, versions
0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and 1.0.2, but with the following
exception: 0.9.0 and above can correctly decompress multiple
concatenated compressed files. 0.1pl2 cannot do this; it will stop
after decompressing just the first file in the stream.
.I bzip2recover
versions prior to 1.0.2 used 32-bit integers to represent
bit positions in compressed files, so they could not handle compressed
files more than 512 megabytes long. Versions 1.0.2 and above use
64-bit ints on some platforms which support them (GNU supported
targets, and Windows). To establish whether or not bzip2recover was
built with such a limitation, run it without arguments. In any event
you can build yourself an unlimited version if you can recompile it
with MaybeUInt64 set to be an unsigned 64-bit integer.
.SH AUTHOR
Julian Seward, jsewardbzip.org.
http://www.bzip.org
The ideas embodied in
.I bzip2
are due to (at least) the following
people: Michael Burrows and David Wheeler (for the block sorting
transformation), David Wheeler (again, for the Huffman coder), Peter
Fenwick (for the structured coding model in the original
.I bzip,
and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
(for the arithmetic coder in the original
.I bzip).
I am much
indebted for their help, support and advice. See the manual in the
source distribution for pointers to sources of documentation. Christian
von Roques encouraged me to look for faster sorting algorithms, so as to
speed up compression. Bela Lubkin encouraged me to improve the
worst-case compression performance.
Donna Robinson XMLised the documentation.
The bz* scripts are derived from those of GNU gzip.
Many people sent patches, helped
with portability problems, lent machines, gave advice and were generally
helpful.

399
contrib/vmap_extractor_v2/stormlib/bzip2/bzip2.1.preformatted
View file

@ -1,399 +0,0 @@
bzip2(1) bzip2(1)
NNAAMMEE
bzip2, bunzip2 − a block‐sorting file compressor, v1.0.3
bzcat − decompresses files to stdout
bzip2recover − recovers data from damaged bzip2 files
SSYYNNOOPPSSIISS
bbzziipp22 [ −−ccddffkkqqssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ]
bbuunnzziipp22 [ −−ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ]
bbzzccaatt [ −−ss ] [ _f_i_l_e_n_a_m_e_s _._._. ]
bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e
DDEESSCCRRIIPPTTIIOONN
_b_z_i_p_2 compresses files using the Burrows‐Wheeler block
sorting text compression algorithm, and Huffman coding.
Compression is generally considerably better than that
achieved by more conventional LZ77/LZ78‐based compressors,
and approaches the performance of the PPM family of sta­
tistical compressors.
The command‐line options are deliberately very similar to
those of _G_N_U _g_z_i_p_, but they are not identical.
_b_z_i_p_2 expects a list of file names to accompany the com­
mand‐line flags. Each file is replaced by a compressed
version of itself, with the name "original_name.bz2".
Each compressed file has the same modification date, per­
missions, and, when possible, ownership as the correspond­
ing original, so that these properties can be correctly
restored at decompression time. File name handling is
naive in the sense that there is no mechanism for preserv­
ing original file names, permissions, ownerships or dates
in filesystems which lack these concepts, or have serious
file name length restrictions, such as MS‐DOS.
_b_z_i_p_2 and _b_u_n_z_i_p_2 will by default not overwrite existing
files. If you want this to happen, specify the −f flag.
If no file names are specified, _b_z_i_p_2 compresses from
standard input to standard output. In this case, _b_z_i_p_2
will decline to write compressed output to a terminal, as
this would be entirely incomprehensible and therefore
pointless.
_b_u_n_z_i_p_2 (or _b_z_i_p_2 _−_d_) decompresses all specified files.
Files which were not created by _b_z_i_p_2 will be detected and
ignored, and a warning issued. _b_z_i_p_2 attempts to guess
the filename for the decompressed file from that of the
compressed file as follows:
filename.bz2 becomes filename
filename.bz becomes filename
filename.tbz2 becomes filename.tar
filename.tbz becomes filename.tar
anyothername becomes anyothername.out
If the file does not end in one of the recognised endings,
_._b_z_2_, _._b_z_, _._t_b_z_2 or _._t_b_z_, _b_z_i_p_2 complains that it cannot
guess the name of the original file, and uses the original
name with _._o_u_t appended.
As with compression, supplying no filenames causes decom­
pression from standard input to standard output.
_b_u_n_z_i_p_2 will correctly decompress a file which is the con­
catenation of two or more compressed files. The result is
the concatenation of the corresponding uncompressed files.
Integrity testing (−t) of concatenated compressed files is
also supported.
You can also compress or decompress files to the standard
output by giving the −c flag. Multiple files may be com­
pressed and decompressed like this. The resulting outputs
are fed sequentially to stdout. Compression of multiple
files in this manner generates a stream containing multi­
ple compressed file representations. Such a stream can be
decompressed correctly only by _b_z_i_p_2 version 0.9.0 or
later. Earlier versions of _b_z_i_p_2 will stop after decom­
pressing the first file in the stream.
_b_z_c_a_t (or _b_z_i_p_2 _‐_d_c_) decompresses all specified files to
the standard output.
_b_z_i_p_2 will read arguments from the environment variables
_B_Z_I_P_2 and _B_Z_I_P_, in that order, and will process them
before any arguments read from the command line. This
gives a convenient way to supply default arguments.
Compression is always performed, even if the compressed
file is slightly larger than the original. Files of less
than about one hundred bytes tend to get larger, since the
compression mechanism has a constant overhead in the
region of 50 bytes. Random data (including the output of
most file compressors) is coded at about 8.05 bits per
byte, giving an expansion of around 0.5%.
As a self‐check for your protection, _b_z_i_p_2 uses 32‐bit
CRCs to make sure that the decompressed version of a file
is identical to the original. This guards against corrup­
tion of the compressed data, and against undetected bugs
in _b_z_i_p_2 (hopefully very unlikely). The chances of data
corruption going undetected is microscopic, about one
chance in four billion for each file processed. Be aware,
though, that the check occurs upon decompression, so it
can only tell you that something is wrong. It can’t help
you recover the original uncompressed data. You can use
_b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files.
Return values: 0 for a normal exit, 1 for environmental
problems (file not found, invalid flags, I/O errors, &c),
2 to indicate a corrupt compressed file, 3 for an internal
consistency error (eg, bug) which caused _b_z_i_p_2 to panic.
OOPPTTIIOONNSS
−−cc ‐‐‐‐ssttddoouutt
Compress or decompress to standard output.
−−dd ‐‐‐‐ddeeccoommpprreessss
Force decompression. _b_z_i_p_2_, _b_u_n_z_i_p_2 and _b_z_c_a_t are
really the same program, and the decision about
what actions to take is done on the basis of which
name is used. This flag overrides that mechanism,
and forces _b_z_i_p_2 to decompress.
−−zz ‐‐‐‐ccoommpprreessss
The complement to −d: forces compression,
regardless of the invocation name.
−−tt ‐‐‐‐tteesstt
Check integrity of the specified file(s), but don’t
decompress them. This really performs a trial
decompression and throws away the result.
−−ff ‐‐‐‐ffoorrccee
Force overwrite of output files. Normally, _b_z_i_p_2
will not overwrite existing output files. Also
forces _b_z_i_p_2 to break hard links to files, which it
otherwise wouldn’t do.
bzip2 normally declines to decompress files which
don’t have the correct magic header bytes. If
forced (‐f), however, it will pass such files
through unmodified. This is how GNU gzip behaves.
−−kk ‐‐‐‐kkeeeepp
Keep (don’t delete) input files during compression
or decompression.
−−ss ‐‐‐‐ssmmaallll
Reduce memory usage, for compression, decompression
and testing. Files are decompressed and tested
using a modified algorithm which only requires 2.5
bytes per block byte. This means any file can be
decompressed in 2300k of memory, albeit at about
half the normal speed.
During compression, −s selects a block size of
200k, which limits memory use to around the same
figure, at the expense of your compression ratio.
In short, if your machine is low on memory (8
megabytes or less), use −s for everything. See
MEMORY MANAGEMENT below.
−−qq ‐‐‐‐qquuiieett
Suppress non‐essential warning messages. Messages
pertaining to I/O errors and other critical events
will not be suppressed.
−−vv ‐‐‐‐vveerrbboossee
Verbose mode ‐‐ show the compression ratio for each
file processed. Further −v’s increase the ver­
bosity level, spewing out lots of information which
is primarily of interest for diagnostic purposes.
−−LL ‐‐‐‐lliicceennssee ‐‐VV ‐‐‐‐vveerrssiioonn
Display the software version, license terms and
conditions.
−−11 ((oorr −−−−ffaasstt)) ttoo −−99 ((oorr −−−−bbeesstt))
Set the block size to 100 k, 200 k .. 900 k when
compressing. Has no effect when decompressing.
See MEMORY MANAGEMENT below. The −−fast and −−best
aliases are primarily for GNU gzip compatibility.
In particular, −−fast doesn’t make things signifi­
cantly faster. And −−best merely selects the
default behaviour.
−−‐‐ Treats all subsequent arguments as file names, even
if they start with a dash. This is so you can han­
dle files with names beginning with a dash, for
example: bzip2 −‐ −myfilename.
−−‐‐rreeppeettiittiivvee‐‐ffaasstt ‐‐‐‐rreeppeettiittiivvee‐‐bbeesstt
These flags are redundant in versions 0.9.5 and
above. They provided some coarse control over the
behaviour of the sorting algorithm in earlier ver­
sions, which was sometimes useful. 0.9.5 and above
have an improved algorithm which renders these
flags irrelevant.
MMEEMMOORRYY MMAANNAAGGEEMMEENNTT
_b_z_i_p_2 compresses large files in blocks. The block size
affects both the compression ratio achieved, and the
amount of memory needed for compression and decompression.
The flags −1 through −9 specify the block size to be
100,000 bytes through 900,000 bytes (the default) respec­
tively. At decompression time, the block size used for
compression is read from the header of the compressed
file, and _b_u_n_z_i_p_2 then allocates itself just enough memory
to decompress the file. Since block sizes are stored in
compressed files, it follows that the flags −1 to −9 are
irrelevant to and so ignored during decompression.
Compression and decompression requirements, in bytes, can
be estimated as:
Compression: 400k + ( 8 x block size )
Decompression: 100k + ( 4 x block size ), or
100k + ( 2.5 x block size )
Larger block sizes give rapidly diminishing marginal
returns. Most of the compression comes from the first two
or three hundred k of block size, a fact worth bearing in
mind when using _b_z_i_p_2 on small machines. It is also
important to appreciate that the decompression memory
requirement is set at compression time by the choice of
block size.
For files compressed with the default 900k block size,
_b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To
support decompression of any file on a 4 megabyte machine,
_b_u_n_z_i_p_2 has an option to decompress using approximately
half this amount of memory, about 2300 kbytes. Decompres­
sion speed is also halved, so you should use this option
only where necessary. The relevant flag is ‐s.
In general, try and use the largest block size memory con­
straints allow, since that maximises the compression
achieved. Compression and decompression speed are virtu­
ally unaffected by block size.
Another significant point applies to files which fit in a
single block ‐‐ that means most files you’d encounter
using a large block size. The amount of real memory
touched is proportional to the size of the file, since the
file is smaller than a block. For example, compressing a
file 20,000 bytes long with the flag ‐9 will cause the
compressor to allocate around 7600k of memory, but only
touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the
decompressor will allocate 3700k but only touch 100k +
20000 * 4 = 180 kbytes.
Here is a table which summarises the maximum memory usage
for different block sizes. Also recorded is the total
compressed size for 14 files of the Calgary Text Compres­
sion Corpus totalling 3,141,622 bytes. This column gives
some feel for how compression varies with block size.
These figures tend to understate the advantage of larger
block sizes for larger files, since the Corpus is domi­
nated by smaller files.
Compress Decompress Decompress Corpus
Flag usage usage ‐s usage Size
‐1 1200k 500k 350k 914704
‐2 2000k 900k 600k 877703
‐3 2800k 1300k 850k 860338
‐4 3600k 1700k 1100k 846899
‐5 4400k 2100k 1350k 845160
‐6 5200k 2500k 1600k 838626
‐7 6100k 2900k 1850k 834096
‐8 6800k 3300k 2100k 828642
‐9 7600k 3700k 2350k 828642
RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS
_b_z_i_p_2 compresses files in blocks, usually 900kbytes long.
Each block is handled independently. If a media or trans­
mission error causes a multi‐block .bz2 file to become
damaged, it may be possible to recover data from the
undamaged blocks in the file.
The compressed representation of each block is delimited
by a 48‐bit pattern, which makes it possible to find the
block boundaries with reasonable certainty. Each block
also carries its own 32‐bit CRC, so damaged blocks can be
distinguished from undamaged ones.
_b_z_i_p_2_r_e_c_o_v_e_r is a simple program whose purpose is to
search for blocks in .bz2 files, and write each block out
into its own .bz2 file. You can then use _b_z_i_p_2 −t to test
the integrity of the resulting files, and decompress those
which are undamaged.
_b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam­
aged file, and writes a number of files
"rec00001file.bz2", "rec00002file.bz2", etc, containing
the extracted blocks. The output filenames are
designed so that the use of wildcards in subsequent pro­
cessing ‐‐ for example, "bzip2 ‐dc rec*file.bz2 > recov­
ered_data" ‐‐ processes the files in the correct order.
_b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2
files, as these will contain many blocks. It is clearly
futile to use it on damaged single‐block files, since a
damaged block cannot be recovered. If you wish to min­
imise any potential data loss through media or transmis­
sion errors, you might consider compressing with a smaller
block size.
PPEERRFFOORRMMAANNCCEE NNOOTTEESS
The sorting phase of compression gathers together similar
strings in the file. Because of this, files containing
very long runs of repeated symbols, like "aabaabaabaab
..." (repeated several hundred times) may compress more
slowly than normal. Versions 0.9.5 and above fare much
better than previous versions in this respect. The ratio
between worst‐case and average‐case compression time is in
the region of 10:1. For previous versions, this figure
was more like 100:1. You can use the −vvvv option to mon­
itor progress in great detail, if you want.
Decompression speed is unaffected by these phenomena.
_b_z_i_p_2 usually allocates several megabytes of memory to
operate in, and then charges all over it in a fairly ran­
dom fashion. This means that performance, both for com­
pressing and decompressing, is largely determined by the
speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the
miss rate have been observed to give disproportionately
large performance improvements. I imagine _b_z_i_p_2 will per­
form best on machines with very large caches.
CCAAVVEEAATTSS
I/O error messages are not as helpful as they could be.
_b_z_i_p_2 tries hard to detect I/O errors and exit cleanly,
but the details of what the problem is sometimes seem
rather misleading.
This manual page pertains to version 1.0.3 of _b_z_i_p_2_. Com­
pressed data created by this version is entirely forwards
and backwards compatible with the previous public
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
1.0.2, but with the following exception: 0.9.0 and above
can correctly decompress multiple concatenated compressed
files. 0.1pl2 cannot do this; it will stop after decom­
pressing just the first file in the stream.
_b_z_i_p_2_r_e_c_o_v_e_r versions prior to 1.0.2 used 32‐bit integers
to represent bit positions in compressed files, so they
could not handle compressed files more than 512 megabytes
long. Versions 1.0.2 and above use 64‐bit ints on some
platforms which support them (GNU supported targets, and
Windows). To establish whether or not bzip2recover was
built with such a limitation, run it without arguments.
In any event you can build yourself an unlimited version
if you can recompile it with MaybeUInt64 set to be an
unsigned 64‐bit integer.
AAUUTTHHOORR
Julian Seward, jsewardbzip.org.
http://www.bzip.org
The ideas embodied in _b_z_i_p_2 are due to (at least) the fol­
lowing people: Michael Burrows and David Wheeler (for the
block sorting transformation), David Wheeler (again, for
the Huffman coder), Peter Fenwick (for the structured cod­
ing model in the original _b_z_i_p_, and many refinements), and
Alistair Moffat, Radford Neal and Ian Witten (for the
arithmetic coder in the original _b_z_i_p_)_. I am much
indebted for their help, support and advice. See the man­
ual in the source distribution for pointers to sources of
documentation. Christian von Roques encouraged me to look
for faster sorting algorithms, so as to speed up compres­
sion. Bela Lubkin encouraged me to improve the worst‐case
compression performance. Donna Robinson XMLised the docu­
mentation. The bz* scripts are derived from those of GNU
gzip. Many people sent patches, helped with portability
problems, lent machines, gave advice and were generally
helpful.
bzip2(1)

2107
contrib/vmap_extractor_v2/stormlib/bzip2/bzip2.c
View file

File diff suppressed because it is too large Load diff

391
contrib/vmap_extractor_v2/stormlib/bzip2/bzip2.txt
View file

@ -1,391 +0,0 @@
NAME
bzip2, bunzip2 - a block-sorting file compressor, v1.0.3
bzcat - decompresses files to stdout
bzip2recover - recovers data from damaged bzip2 files
SYNOPSIS
bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ]
bunzip2 [ -fkvsVL ] [ filenames ... ]
bzcat [ -s ] [ filenames ... ]
bzip2recover filename
DESCRIPTION
bzip2 compresses files using the Burrows-Wheeler block
sorting text compression algorithm, and Huffman coding.
Compression is generally considerably better than that
achieved by more conventional LZ77/LZ78-based compressors,
and approaches the performance of the PPM family of sta-
tistical compressors.
The command-line options are deliberately very similar to
those of GNU gzip, but they are not identical.
bzip2 expects a list of file names to accompany the com-
mand-line flags. Each file is replaced by a compressed
version of itself, with the name "original_name.bz2".
Each compressed file has the same modification date, per-
missions, and, when possible, ownership as the correspond-
ing original, so that these properties can be correctly
restored at decompression time. File name handling is
naive in the sense that there is no mechanism for preserv-
ing original file names, permissions, ownerships or dates
in filesystems which lack these concepts, or have serious
file name length restrictions, such as MS-DOS.
bzip2 and bunzip2 will by default not overwrite existing
files. If you want this to happen, specify the -f flag.
If no file names are specified, bzip2 compresses from
standard input to standard output. In this case, bzip2
will decline to write compressed output to a terminal, as
this would be entirely incomprehensible and therefore
pointless.
bunzip2 (or bzip2 -d) decompresses all specified files.
Files which were not created by bzip2 will be detected and
ignored, and a warning issued. bzip2 attempts to guess
the filename for the decompressed file from that of the
compressed file as follows:
filename.bz2 becomes filename
filename.bz becomes filename
filename.tbz2 becomes filename.tar
filename.tbz becomes filename.tar
anyothername becomes anyothername.out
If the file does not end in one of the recognised endings,
.bz2, .bz, .tbz2 or .tbz, bzip2 complains that it cannot
guess the name of the original file, and uses the original
name with .out appended.
As with compression, supplying no filenames causes decom-
pression from standard input to standard output.
bunzip2 will correctly decompress a file which is the con-
catenation of two or more compressed files. The result is
the concatenation of the corresponding uncompressed files.
Integrity testing (-t) of concatenated compressed files is
also supported.
You can also compress or decompress files to the standard
output by giving the -c flag. Multiple files may be com-
pressed and decompressed like this. The resulting outputs
are fed sequentially to stdout. Compression of multiple
files in this manner generates a stream containing multi-
ple compressed file representations. Such a stream can be
decompressed correctly only by bzip2 version 0.9.0 or
later. Earlier versions of bzip2 will stop after decom-
pressing the first file in the stream.
bzcat (or bzip2 -dc) decompresses all specified files to
the standard output.
bzip2 will read arguments from the environment variables
BZIP2 and BZIP, in that order, and will process them
before any arguments read from the command line. This
gives a convenient way to supply default arguments.
Compression is always performed, even if the compressed
file is slightly larger than the original. Files of less
than about one hundred bytes tend to get larger, since the
compression mechanism has a constant overhead in the
region of 50 bytes. Random data (including the output of
most file compressors) is coded at about 8.05 bits per
byte, giving an expansion of around 0.5%.
As a self-check for your protection, bzip2 uses 32-bit
CRCs to make sure that the decompressed version of a file
is identical to the original. This guards against corrup-
tion of the compressed data, and against undetected bugs
in bzip2 (hopefully very unlikely). The chances of data
corruption going undetected is microscopic, about one
chance in four billion for each file processed. Be aware,
though, that the check occurs upon decompression, so it
can only tell you that something is wrong. It can't help
you recover the original uncompressed data. You can use
bzip2recover to try to recover data from damaged files.
Return values: 0 for a normal exit, 1 for environmental
problems (file not found, invalid flags, I/O errors, &c),
2 to indicate a corrupt compressed file, 3 for an internal
consistency error (eg, bug) which caused bzip2 to panic.
OPTIONS
-c --stdout
Compress or decompress to standard output.
-d --decompress
Force decompression. bzip2, bunzip2 and bzcat are
really the same program, and the decision about
what actions to take is done on the basis of which
name is used. This flag overrides that mechanism,
and forces bzip2 to decompress.
-z --compress
The complement to -d: forces compression,
regardless of the invocation name.
-t --test
Check integrity of the specified file(s), but don't
decompress them. This really performs a trial
decompression and throws away the result.
-f --force
Force overwrite of output files. Normally, bzip2
will not overwrite existing output files. Also
forces bzip2 to break hard links to files, which it
otherwise wouldn't do.
bzip2 normally declines to decompress files which
don't have the correct magic header bytes. If
forced (-f), however, it will pass such files
through unmodified. This is how GNU gzip behaves.
-k --keep
Keep (don't delete) input files during compression
or decompression.
-s --small
Reduce memory usage, for compression, decompression
and testing. Files are decompressed and tested
using a modified algorithm which only requires 2.5
bytes per block byte. This means any file can be
decompressed in 2300k of memory, albeit at about
half the normal speed.
During compression, -s selects a block size of
200k, which limits memory use to around the same
figure, at the expense of your compression ratio.
In short, if your machine is low on memory (8
megabytes or less), use -s for everything. See
MEMORY MANAGEMENT below.
-q --quiet
Suppress non-essential warning messages. Messages
pertaining to I/O errors and other critical events
will not be suppressed.
-v --verbose
Verbose mode -- show the compression ratio for each
file processed. Further -v's increase the ver-
bosity level, spewing out lots of information which
is primarily of interest for diagnostic purposes.
-L --license -V --version
Display the software version, license terms and
conditions.
-1 (or --fast) to -9 (or --best)
Set the block size to 100 k, 200 k .. 900 k when
compressing. Has no effect when decompressing.
See MEMORY MANAGEMENT below. The --fast and --best
aliases are primarily for GNU gzip compatibility.
In particular, --fast doesn't make things signifi-
cantly faster. And --best merely selects the
default behaviour.
-- Treats all subsequent arguments as file names, even
if they start with a dash. This is so you can han-
dle files with names beginning with a dash, for
example: bzip2 -- -myfilename.
--repetitive-fast --repetitive-best
These flags are redundant in versions 0.9.5 and
above. They provided some coarse control over the
behaviour of the sorting algorithm in earlier ver-
sions, which was sometimes useful. 0.9.5 and above
have an improved algorithm which renders these
flags irrelevant.
MEMORY MANAGEMENT
bzip2 compresses large files in blocks. The block size
affects both the compression ratio achieved, and the
amount of memory needed for compression and decompression.
The flags -1 through -9 specify the block size to be
100,000 bytes through 900,000 bytes (the default) respec-
tively. At decompression time, the block size used for
compression is read from the header of the compressed
file, and bunzip2 then allocates itself just enough memory
to decompress the file. Since block sizes are stored in
compressed files, it follows that the flags -1 to -9 are
irrelevant to and so ignored during decompression.
Compression and decompression requirements, in bytes, can
be estimated as:
Compression: 400k + ( 8 x block size )
Decompression: 100k + ( 4 x block size ), or
100k + ( 2.5 x block size )
Larger block sizes give rapidly diminishing marginal
returns. Most of the compression comes from the first two
or three hundred k of block size, a fact worth bearing in
mind when using bzip2 on small machines. It is also
important to appreciate that the decompression memory
requirement is set at compression time by the choice of
block size.
For files compressed with the default 900k block size,
bunzip2 will require about 3700 kbytes to decompress. To
support decompression of any file on a 4 megabyte machine,
bunzip2 has an option to decompress using approximately
half this amount of memory, about 2300 kbytes. Decompres-
sion speed is also halved, so you should use this option
only where necessary. The relevant flag is -s.
In general, try and use the largest block size memory con-
straints allow, since that maximises the compression
achieved. Compression and decompression speed are virtu-
ally unaffected by block size.
Another significant point applies to files which fit in a
single block -- that means most files you'd encounter
using a large block size. The amount of real memory
touched is proportional to the size of the file, since the
file is smaller than a block. For example, compressing a
file 20,000 bytes long with the flag -9 will cause the
compressor to allocate around 7600k of memory, but only
touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the
decompressor will allocate 3700k but only touch 100k +
20000 * 4 = 180 kbytes.
Here is a table which summarises the maximum memory usage
for different block sizes. Also recorded is the total
compressed size for 14 files of the Calgary Text Compres-
sion Corpus totalling 3,141,622 bytes. This column gives
some feel for how compression varies with block size.
These figures tend to understate the advantage of larger
block sizes for larger files, since the Corpus is domi-
nated by smaller files.
Compress Decompress Decompress Corpus
Flag usage usage -s usage Size
-1 1200k 500k 350k 914704
-2 2000k 900k 600k 877703
-3 2800k 1300k 850k 860338
-4 3600k 1700k 1100k 846899
-5 4400k 2100k 1350k 845160
-6 5200k 2500k 1600k 838626
-7 6100k 2900k 1850k 834096
-8 6800k 3300k 2100k 828642
-9 7600k 3700k 2350k 828642
RECOVERING DATA FROM DAMAGED FILES
bzip2 compresses files in blocks, usually 900kbytes long.
Each block is handled independently. If a media or trans-
mission error causes a multi-block .bz2 file to become
damaged, it may be possible to recover data from the
undamaged blocks in the file.
The compressed representation of each block is delimited
by a 48-bit pattern, which makes it possible to find the
block boundaries with reasonable certainty. Each block
also carries its own 32-bit CRC, so damaged blocks can be
distinguished from undamaged ones.
bzip2recover is a simple program whose purpose is to
search for blocks in .bz2 files, and write each block out
into its own .bz2 file. You can then use bzip2 -t to test
the integrity of the resulting files, and decompress those
which are undamaged.
bzip2recover takes a single argument, the name of the dam-
aged file, and writes a number of files
"rec00001file.bz2", "rec00002file.bz2", etc, containing
the extracted blocks. The output filenames are
designed so that the use of wildcards in subsequent pro-
cessing -- for example, "bzip2 -dc rec*file.bz2 > recov-
ered_data" -- processes the files in the correct order.
bzip2recover should be of most use dealing with large .bz2
files, as these will contain many blocks. It is clearly
futile to use it on damaged single-block files, since a
damaged block cannot be recovered. If you wish to min-
imise any potential data loss through media or transmis-
sion errors, you might consider compressing with a smaller
block size.
PERFORMANCE NOTES
The sorting phase of compression gathers together similar
strings in the file. Because of this, files containing
very long runs of repeated symbols, like "aabaabaabaab
..." (repeated several hundred times) may compress more
slowly than normal. Versions 0.9.5 and above fare much
better than previous versions in this respect. The ratio
between worst-case and average-case compression time is in
the region of 10:1. For previous versions, this figure
was more like 100:1. You can use the -vvvv option to mon-
itor progress in great detail, if you want.
Decompression speed is unaffected by these phenomena.
bzip2 usually allocates several megabytes of memory to
operate in, and then charges all over it in a fairly ran-
dom fashion. This means that performance, both for com-
pressing and decompressing, is largely determined by the
speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the
miss rate have been observed to give disproportionately
large performance improvements. I imagine bzip2 will per-
form best on machines with very large caches.
CAVEATS
I/O error messages are not as helpful as they could be.
bzip2 tries hard to detect I/O errors and exit cleanly,
but the details of what the problem is sometimes seem
rather misleading.
This manual page pertains to version 1.0.3 of bzip2. Com-
pressed data created by this version is entirely forwards
and backwards compatible with the previous public
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1 and
1.0.2, but with the following exception: 0.9.0 and above
can correctly decompress multiple concatenated compressed
files. 0.1pl2 cannot do this; it will stop after decom-
pressing just the first file in the stream.
bzip2recover versions prior to 1.0.2 used 32-bit integers
to represent bit positions in compressed files, so they
could not handle compressed files more than 512 megabytes
long. Versions 1.0.2 and above use 64-bit ints on some
platforms which support them (GNU supported targets, and
Windows). To establish whether or not bzip2recover was
built with such a limitation, run it without arguments.
In any event you can build yourself an unlimited version
if you can recompile it with MaybeUInt64 set to be an
unsigned 64-bit integer.
AUTHOR
Julian Seward, jsewardbzip.org.
http://www.bzip.org
The ideas embodied in bzip2 are due to (at least) the fol-
lowing people: Michael Burrows and David Wheeler (for the
block sorting transformation), David Wheeler (again, for
the Huffman coder), Peter Fenwick (for the structured cod-
ing model in the original bzip, and many refinements), and
Alistair Moffat, Radford Neal and Ian Witten (for the
arithmetic coder in the original bzip). I am much
indebted for their help, support and advice. See the man-
ual in the source distribution for pointers to sources of
documentation. Christian von Roques encouraged me to look
for faster sorting algorithms, so as to speed up compres-
sion. Bela Lubkin encouraged me to improve the worst-case
compression performance. Donna Robinson XMLised the docu-
mentation. The bz* scripts are derived from those of GNU
gzip. Many people sent patches, helped with portability
problems, lent machines, gave advice and were generally
helpful.

546
contrib/vmap_extractor_v2/stormlib/bzip2/bzip2recover.c
View file

@ -1,546 +0,0 @@
/*-----------------------------------------------------------*/
/*--- Block recoverer program for bzip2 ---*/
/*--- bzip2recover.c ---*/
/*-----------------------------------------------------------*/
/*--
This program is bzip2recover, a program to attempt data
salvage from damaged files created by the accompanying
bzip2-1.0.3 program.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0.3 of 15 February 2005
--*/
/*--
This program is a complete hack and should be rewritten
properly. It isn't very complicated.
--*/
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <string.h>
/* This program records bit locations in the file to be recovered.
That means that if 64-bit ints are not supported, we will not
be able to recover .bz2 files over 512MB (2^32 bits) long.
On GNU supported platforms, we take advantage of the 64-bit
int support to circumvent this problem. Ditto MSVC.
This change occurred in version 1.0.2; all prior versions have
the 512MB limitation.
*/
#ifdef __GNUC__
typedef unsigned long long int MaybeUInt64;
# define MaybeUInt64_FMT "%Lu"
#else
#ifdef _MSC_VER
typedef unsigned __int64 MaybeUInt64;
# define MaybeUInt64_FMT "%I64u"
#else
typedef unsigned int MaybeUInt64;
# define MaybeUInt64_FMT "%u"
#endif
#endif
typedef unsigned int UInt32;
typedef int Int32;
typedef unsigned char UChar;
typedef char Char;
typedef unsigned char Bool;
#define True ((Bool)1)
#define False ((Bool)0)
#define BZ_MAX_FILENAME 2000
Char inFileName[BZ_MAX_FILENAME];
Char outFileName[BZ_MAX_FILENAME];
Char progName[BZ_MAX_FILENAME];
MaybeUInt64 bytesOut = 0;
MaybeUInt64 bytesIn = 0;
/*---------------------------------------------------*/
/*--- Header bytes ---*/
/*---------------------------------------------------*/
#define BZ_HDR_B 0x42 /* 'B' */
#define BZ_HDR_Z 0x5a /* 'Z' */
#define BZ_HDR_h 0x68 /* 'h' */
#define BZ_HDR_0 0x30 /* '0' */
/*---------------------------------------------------*/
/*--- I/O errors ---*/
/*---------------------------------------------------*/
/*---------------------------------------------*/
void readError ( void )
{
fprintf ( stderr,
"%s: I/O error reading `%s', possible reason follows.\n",
progName, inFileName );
perror ( progName );
fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------*/
void writeError ( void )
{
fprintf ( stderr,
"%s: I/O error reading `%s', possible reason follows.\n",
progName, inFileName );
perror ( progName );
fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------*/
void mallocFail ( Int32 n )
{
fprintf ( stderr,
"%s: malloc failed on request for %d bytes.\n",
progName, n );
fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------*/
void tooManyBlocks ( Int32 max_handled_blocks )
{
fprintf ( stderr,
"%s: `%s' appears to contain more than %d blocks\n",
progName, inFileName, max_handled_blocks );
fprintf ( stderr,
"%s: and cannot be handled. To fix, increase\n",
progName );
fprintf ( stderr,
"%s: BZ_MAX_HANDLED_BLOCKS in bzip2recover.c, and recompile.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------------*/
/*--- Bit stream I/O ---*/
/*---------------------------------------------------*/
typedef
struct {
FILE* handle;
Int32 buffer;
Int32 buffLive;
Char mode;
}
BitStream;
/*---------------------------------------------*/
BitStream* bsOpenReadStream ( FILE* stream )
{
BitStream *bs = malloc ( sizeof(BitStream) );
if (bs == NULL) mallocFail ( sizeof(BitStream) );
bs->handle = stream;
bs->buffer = 0;
bs->buffLive = 0;
bs->mode = 'r';
return bs;
}
/*---------------------------------------------*/
BitStream* bsOpenWriteStream ( FILE* stream )
{
BitStream *bs = malloc ( sizeof(BitStream) );
if (bs == NULL) mallocFail ( sizeof(BitStream) );
bs->handle = stream;
bs->buffer = 0;
bs->buffLive = 0;
bs->mode = 'w';
return bs;
}
/*---------------------------------------------*/
void bsPutBit ( BitStream* bs, Int32 bit )
{
if (bs->buffLive == 8) {
Int32 retVal = putc ( (UChar) bs->buffer, bs->handle );
if (retVal == EOF) writeError();
bytesOut++;
bs->buffLive = 1;
bs->buffer = bit & 0x1;
} else {
bs->buffer = ( (bs->buffer << 1) | (bit & 0x1) );
bs->buffLive++;
};
}
/*---------------------------------------------*/
/*--
Returns 0 or 1, or 2 to indicate EOF.
--*/
Int32 bsGetBit ( BitStream* bs )
{
if (bs->buffLive > 0) {
bs->buffLive --;
return ( ((bs->buffer) >> (bs->buffLive)) & 0x1 );
} else {
Int32 retVal = getc ( bs->handle );
if ( retVal == EOF ) {
if (errno != 0) readError();
return 2;
}
bs->buffLive = 7;
bs->buffer = retVal;
return ( ((bs->buffer) >> 7) & 0x1 );
}
}
/*---------------------------------------------*/
void bsClose ( BitStream* bs )
{
Int32 retVal;
if ( bs->mode == 'w' ) {
while ( bs->buffLive < 8 ) {
bs->buffLive++;
bs->buffer <<= 1;
};
retVal = putc ( (UChar) (bs->buffer), bs->handle );
if (retVal == EOF) writeError();
bytesOut++;
retVal = fflush ( bs->handle );
if (retVal == EOF) writeError();
}
retVal = fclose ( bs->handle );
if (retVal == EOF) {
if (bs->mode == 'w') writeError(); else readError();
}
free ( bs );
}
/*---------------------------------------------*/
void bsPutUChar ( BitStream* bs, UChar c )
{
Int32 i;
for (i = 7; i >= 0; i--)
bsPutBit ( bs, (((UInt32) c) >> i) & 0x1 );
}
/*---------------------------------------------*/
void bsPutUInt32 ( BitStream* bs, UInt32 c )
{
Int32 i;
for (i = 31; i >= 0; i--)
bsPutBit ( bs, (c >> i) & 0x1 );
}
/*---------------------------------------------*/
Bool endsInBz2 ( Char* name )
{
Int32 n = strlen ( name );
if (n <= 4) return False;
return
(name[n-4] == '.' &&
name[n-3] == 'b' &&
name[n-2] == 'z' &&
name[n-1] == '2');
}
/*---------------------------------------------------*/
/*--- ---*/
/*---------------------------------------------------*/
/* This logic isn't really right when it comes to Cygwin. */
#ifdef _WIN32
# define BZ_SPLIT_SYM '\\' /* path splitter on Windows platform */
#else
# define BZ_SPLIT_SYM '/' /* path splitter on Unix platform */
#endif
#define BLOCK_HEADER_HI 0x00003141UL
#define BLOCK_HEADER_LO 0x59265359UL
#define BLOCK_ENDMARK_HI 0x00001772UL
#define BLOCK_ENDMARK_LO 0x45385090UL
/* Increase if necessary. However, a .bz2 file with > 50000 blocks
would have an uncompressed size of at least 40GB, so the chances
are low you'll need to up this.
*/
#define BZ_MAX_HANDLED_BLOCKS 50000
MaybeUInt64 bStart [BZ_MAX_HANDLED_BLOCKS];
MaybeUInt64 bEnd [BZ_MAX_HANDLED_BLOCKS];
MaybeUInt64 rbStart[BZ_MAX_HANDLED_BLOCKS];
MaybeUInt64 rbEnd [BZ_MAX_HANDLED_BLOCKS];
Int32 main ( Int32 argc, Char** argv )
{
FILE* inFile;
FILE* outFile;
BitStream* bsIn, *bsWr;
Int32 b, wrBlock, currBlock, rbCtr;
MaybeUInt64 bitsRead;
UInt32 buffHi, buffLo, blockCRC;
Char* p;
strcpy ( progName, argv[0] );
inFileName[0] = outFileName[0] = 0;
fprintf ( stderr,
"bzip2recover 1.0.3: extracts blocks from damaged .bz2 files.\n" );
if (argc != 2) {
fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
progName, progName );
switch (sizeof(MaybeUInt64)) {
case 8:
fprintf(stderr,
"\trestrictions on size of recovered file: None\n");
break;
case 4:
fprintf(stderr,
"\trestrictions on size of recovered file: 512 MB\n");
fprintf(stderr,
"\tto circumvent, recompile with MaybeUInt64 as an\n"
"\tunsigned 64-bit int.\n");
break;
default:
fprintf(stderr,
"\tsizeof(MaybeUInt64) is not 4 or 8 -- "
"configuration error.\n");
break;
}
exit(1);
}
if (strlen(argv[1]) >= BZ_MAX_FILENAME-20) {
fprintf ( stderr,
"%s: supplied filename is suspiciously (>= %d chars) long. Bye!\n",
progName, (int)strlen(argv[1]) );
exit(1);
}
strcpy ( inFileName, argv[1] );
inFile = fopen ( inFileName, "rb" );
if (inFile == NULL) {
fprintf ( stderr, "%s: can't read `%s'\n", progName, inFileName );
exit(1);
}
bsIn = bsOpenReadStream ( inFile );
fprintf ( stderr, "%s: searching for block boundaries ...\n", progName );
bitsRead = 0;
buffHi = buffLo = 0;
currBlock = 0;
bStart[currBlock] = 0;
rbCtr = 0;
while (True) {
b = bsGetBit ( bsIn );
bitsRead++;
if (b == 2) {
if (bitsRead >= bStart[currBlock] &&
(bitsRead - bStart[currBlock]) >= 40) {
bEnd[currBlock] = bitsRead-1;
if (currBlock > 0)
fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT
" to " MaybeUInt64_FMT " (incomplete)\n",
currBlock, bStart[currBlock], bEnd[currBlock] );
} else
currBlock--;
break;
}
buffHi = (buffHi << 1) | (buffLo >> 31);
buffLo = (buffLo << 1) | (b & 1);
if ( ( (buffHi & 0x0000ffff) == BLOCK_HEADER_HI
&& buffLo == BLOCK_HEADER_LO)
||
( (buffHi & 0x0000ffff) == BLOCK_ENDMARK_HI
&& buffLo == BLOCK_ENDMARK_LO)
) {
if (bitsRead > 49) {
bEnd[currBlock] = bitsRead-49;
} else {
bEnd[currBlock] = 0;
}
if (currBlock > 0 &&
(bEnd[currBlock] - bStart[currBlock]) >= 130) {
fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT
" to " MaybeUInt64_FMT "\n",
rbCtr+1, bStart[currBlock], bEnd[currBlock] );
rbStart[rbCtr] = bStart[currBlock];
rbEnd[rbCtr] = bEnd[currBlock];
rbCtr++;
}
if (currBlock >= BZ_MAX_HANDLED_BLOCKS)
tooManyBlocks(BZ_MAX_HANDLED_BLOCKS);
currBlock++;
bStart[currBlock] = bitsRead;
}
}
bsClose ( bsIn );
/*-- identified blocks run from 1 to rbCtr inclusive. --*/
if (rbCtr < 1) {
fprintf ( stderr,
"%s: sorry, I couldn't find any block boundaries.\n",
progName );
exit(1);
};
fprintf ( stderr, "%s: splitting into blocks\n", progName );
inFile = fopen ( inFileName, "rb" );
if (inFile == NULL) {
fprintf ( stderr, "%s: can't open `%s'\n", progName, inFileName );
exit(1);
}
bsIn = bsOpenReadStream ( inFile );
/*-- placate gcc's dataflow analyser --*/
blockCRC = 0; bsWr = 0;
bitsRead = 0;
outFile = NULL;
wrBlock = 0;
while (True) {
b = bsGetBit(bsIn);
if (b == 2) break;
buffHi = (buffHi << 1) | (buffLo >> 31);
buffLo = (buffLo << 1) | (b & 1);
if (bitsRead == 47+rbStart[wrBlock])
blockCRC = (buffHi << 16) | (buffLo >> 16);
if (outFile != NULL && bitsRead >= rbStart[wrBlock]
&& bitsRead <= rbEnd[wrBlock]) {
bsPutBit ( bsWr, b );
}
bitsRead++;
if (bitsRead == rbEnd[wrBlock]+1) {
if (outFile != NULL) {
bsPutUChar ( bsWr, 0x17 ); bsPutUChar ( bsWr, 0x72 );
bsPutUChar ( bsWr, 0x45 ); bsPutUChar ( bsWr, 0x38 );
bsPutUChar ( bsWr, 0x50 ); bsPutUChar ( bsWr, 0x90 );
bsPutUInt32 ( bsWr, blockCRC );
bsClose ( bsWr );
}
if (wrBlock >= rbCtr) break;
wrBlock++;
} else
if (bitsRead == rbStart[wrBlock]) {
/* Create the output file name, correctly handling leading paths.
(31.10.2001 by Sergey E. Kusikov) */
Char* split;
Int32 ofs, k;
for (k = 0; k < BZ_MAX_FILENAME; k++)
outFileName[k] = 0;
strcpy (outFileName, inFileName);
split = strrchr (outFileName, BZ_SPLIT_SYM);
if (split == NULL) {
split = outFileName;
} else {
++split;
}
/* Now split points to the start of the basename. */
ofs = split - outFileName;
sprintf (split, "rec%5d", wrBlock+1);
for (p = split; *p != 0; p++) if (*p == ' ') *p = '0';
strcat (outFileName, inFileName + ofs);
if ( !endsInBz2(outFileName)) strcat ( outFileName, ".bz2" );
fprintf ( stderr, " writing block %d to `%s' ...\n",
wrBlock+1, outFileName );
outFile = fopen ( outFileName, "wb" );
if (outFile == NULL) {
fprintf ( stderr, "%s: can't write `%s'\n",
progName, outFileName );
exit(1);
}
bsWr = bsOpenWriteStream ( outFile );
bsPutUChar ( bsWr, BZ_HDR_B );
bsPutUChar ( bsWr, BZ_HDR_Z );
bsPutUChar ( bsWr, BZ_HDR_h );
bsPutUChar ( bsWr, BZ_HDR_0 + 9 );
bsPutUChar ( bsWr, 0x31 ); bsPutUChar ( bsWr, 0x41 );
bsPutUChar ( bsWr, 0x59 ); bsPutUChar ( bsWr, 0x26 );
bsPutUChar ( bsWr, 0x53 ); bsPutUChar ( bsWr, 0x59 );
}
}
fprintf ( stderr, "%s: finished\n", progName );
return 0;
}
/*-----------------------------------------------------------*/
/*--- end bzip2recover.c ---*/
/*-----------------------------------------------------------*/

1617
contrib/vmap_extractor_v2/stormlib/bzip2/bzlib.c
View file

File diff suppressed because it is too large Load diff

323
contrib/vmap_extractor_v2/stormlib/bzip2/bzlib.h
View file

@ -1,323 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Public header file for the library. ---*/
/*--- bzlib.h ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
#ifndef _BZLIB_H
#define _BZLIB_H
#ifdef __cplusplus
extern "C" {
#endif
#define BZ_RUN 0
#define BZ_FLUSH 1
#define BZ_FINISH 2
#define BZ_OK 0
#define BZ_RUN_OK 1
#define BZ_FLUSH_OK 2
#define BZ_FINISH_OK 3
#define BZ_STREAM_END 4
#define BZ_SEQUENCE_ERROR (-1)
#define BZ_PARAM_ERROR (-2)
#define BZ_MEM_ERROR (-3)
#define BZ_DATA_ERROR (-4)
#define BZ_DATA_ERROR_MAGIC (-5)
#define BZ_IO_ERROR (-6)
#define BZ_UNEXPECTED_EOF (-7)
#define BZ_OUTBUFF_FULL (-8)
#define BZ_CONFIG_ERROR (-9)
typedef
struct {
char *next_in;
unsigned int avail_in;
unsigned int total_in_lo32;
unsigned int total_in_hi32;
char *next_out;
unsigned int avail_out;
unsigned int total_out_lo32;
unsigned int total_out_hi32;
void *state;
void *(*bzalloc)(void *,int,int);
void (*bzfree)(void *,void *);
void *opaque;
}
bz_stream;
#ifndef BZ_IMPORT
#define BZ_EXPORT
#endif
#ifndef BZ_NO_STDIO
/* Need a definitition for FILE */
#include <stdio.h>
#endif
#ifdef _WIN32
# include <windows.h>
# ifdef small
/* windows.h define small to char */
# undef small
# endif
# ifdef BZ_EXPORT
# define BZ_API(func) WINAPI func
# define BZ_EXTERN extern
# else
/* import windows dll dynamically */
# define BZ_API(func) (WINAPI * func)
# define BZ_EXTERN
# endif
#else
# define BZ_API(func) func
# define BZ_EXTERN extern
#endif
/*-- Core (low-level) library functions --*/
BZ_EXTERN int BZ_API(BZ2_bzCompressInit) (
bz_stream* strm,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN int BZ_API(BZ2_bzCompress) (
bz_stream* strm,
int action
);
BZ_EXTERN int BZ_API(BZ2_bzCompressEnd) (
bz_stream* strm
);
BZ_EXTERN int BZ_API(BZ2_bzDecompressInit) (
bz_stream *strm,
int verbosity,
int small
);
BZ_EXTERN int BZ_API(BZ2_bzDecompress) (
bz_stream* strm
);
BZ_EXTERN int BZ_API(BZ2_bzDecompressEnd) (
bz_stream *strm
);
/*-- High(er) level library functions --*/
#ifndef BZ_NO_STDIO
#define BZ_MAX_UNUSED 5000
typedef void BZFILE;
BZ_EXTERN BZFILE* BZ_API(BZ2_bzReadOpen) (
int* bzerror,
FILE* f,
int verbosity,
int small,
void* unused,
int nUnused
);
BZ_EXTERN void BZ_API(BZ2_bzReadClose) (
int* bzerror,
BZFILE* b
);
BZ_EXTERN void BZ_API(BZ2_bzReadGetUnused) (
int* bzerror,
BZFILE* b,
void** unused,
int* nUnused
);
BZ_EXTERN int BZ_API(BZ2_bzRead) (
int* bzerror,
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN BZFILE* BZ_API(BZ2_bzWriteOpen) (
int* bzerror,
FILE* f,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN void BZ_API(BZ2_bzWrite) (
int* bzerror,
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN void BZ_API(BZ2_bzWriteClose) (
int* bzerror,
BZFILE* b,
int abandon,
unsigned int* nbytes_in,
unsigned int* nbytes_out
);
BZ_EXTERN void BZ_API(BZ2_bzWriteClose64) (
int* bzerror,
BZFILE* b,
int abandon,
unsigned int* nbytes_in_lo32,
unsigned int* nbytes_in_hi32,
unsigned int* nbytes_out_lo32,
unsigned int* nbytes_out_hi32
);
#endif
/*-- Utility functions --*/
BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffCompress) (
char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffDecompress) (
char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int small,
int verbosity
);
/*--
Code contributed by Yoshioka Tsuneo
(QWF00133@niftyserve.or.jp/tsuneo-y@is.aist-nara.ac.jp),
to support better zlib compatibility.
This code is not _officially_ part of libbzip2 (yet);
I haven't tested it, documented it, or considered the
threading-safeness of it.
If this code breaks, please contact both Yoshioka and me.
--*/
BZ_EXTERN const char * BZ_API(BZ2_bzlibVersion) (
void
);
#ifndef BZ_NO_STDIO
BZ_EXTERN BZFILE * BZ_API(BZ2_bzopen) (
const char *path,
const char *mode
);
BZ_EXTERN BZFILE * BZ_API(BZ2_bzdopen) (
int fd,
const char *mode
);
BZ_EXTERN int BZ_API(BZ2_bzread) (
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN int BZ_API(BZ2_bzwrite) (
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN int BZ_API(BZ2_bzflush) (
BZFILE* b
);
BZ_EXTERN void BZ_API(BZ2_bzclose) (
BZFILE* b
);
BZ_EXTERN const char * BZ_API(BZ2_bzerror) (
BZFILE *b,
int *errnum
);
#endif
#ifdef __cplusplus
}
#endif
#endif
/*-------------------------------------------------------------*/
/*--- end bzlib.h ---*/
/*-------------------------------------------------------------*/

537
contrib/vmap_extractor_v2/stormlib/bzip2/bzlib_private.h
View file

@ -1,537 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Private header file for the library. ---*/
/*--- bzlib_private.h ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
#ifndef _BZLIB_PRIVATE_H
#define _BZLIB_PRIVATE_H
#include <stdlib.h>
#ifndef BZ_NO_STDIO
#include <stdio.h>
#include <ctype.h>
#include <string.h>
#endif
#include "bzlib.h"
/*-- General stuff. --*/
#define BZ_VERSION "1.0.3, 15-Feb-2005"
typedef char Char;
typedef unsigned char Bool;
typedef unsigned char UChar;
typedef int Int32;
typedef unsigned int UInt32;
typedef short Int16;
typedef unsigned short UInt16;
#define True ((Bool)1)
#define False ((Bool)0)
#ifndef __GNUC__
#define __inline__ /* */
#endif
#ifndef BZ_NO_STDIO
extern void BZ2_bz__AssertH__fail ( int errcode );
#define AssertH(cond,errcode) \
{ if (!(cond)) BZ2_bz__AssertH__fail ( errcode ); }
#if BZ_DEBUG
#define AssertD(cond,msg) \
{ if (!(cond)) { \
fprintf ( stderr, \
"\n\nlibbzip2(debug build): internal error\n\t%s\n", msg );\
exit(1); \
}}
#else
#define AssertD(cond,msg) /* */
#endif
#define VPrintf0(zf) \
fprintf(stderr,zf)
#define VPrintf1(zf,za1) \
fprintf(stderr,zf,za1)
#define VPrintf2(zf,za1,za2) \
fprintf(stderr,zf,za1,za2)
#define VPrintf3(zf,za1,za2,za3) \
fprintf(stderr,zf,za1,za2,za3)
#define VPrintf4(zf,za1,za2,za3,za4) \
fprintf(stderr,zf,za1,za2,za3,za4)
#define VPrintf5(zf,za1,za2,za3,za4,za5) \
fprintf(stderr,zf,za1,za2,za3,za4,za5)
#else
extern void bz_internal_error ( int errcode );
#define AssertH(cond,errcode) \
{ if (!(cond)) bz_internal_error ( errcode ); }
#define AssertD(cond,msg) /* */
#define VPrintf0(zf) /* */
#define VPrintf1(zf,za1) /* */
#define VPrintf2(zf,za1,za2) /* */
#define VPrintf3(zf,za1,za2,za3) /* */
#define VPrintf4(zf,za1,za2,za3,za4) /* */
#define VPrintf5(zf,za1,za2,za3,za4,za5) /* */
#endif
#define BZALLOC(nnn) (strm->bzalloc)(strm->opaque,(nnn),1)
#define BZFREE(ppp) (strm->bzfree)(strm->opaque,(ppp))
/*-- Header bytes. --*/
#define BZ_HDR_B 0x42 /* 'B' */
#define BZ_HDR_Z 0x5a /* 'Z' */
#define BZ_HDR_h 0x68 /* 'h' */
#define BZ_HDR_0 0x30 /* '0' */
/*-- Constants for the back end. --*/
#define BZ_MAX_ALPHA_SIZE 258
#define BZ_MAX_CODE_LEN 23
#define BZ_RUNA 0
#define BZ_RUNB 1
#define BZ_N_GROUPS 6
#define BZ_G_SIZE 50
#define BZ_N_ITERS 4
#define BZ_MAX_SELECTORS (2 + (900000 / BZ_G_SIZE))
/*-- Stuff for randomising repetitive blocks. --*/
extern Int32 BZ2_rNums[512];
#define BZ_RAND_DECLS \
Int32 rNToGo; \
Int32 rTPos \
#define BZ_RAND_INIT_MASK \
s->rNToGo = 0; \
s->rTPos = 0 \
#define BZ_RAND_MASK ((s->rNToGo == 1) ? 1 : 0)
#define BZ_RAND_UPD_MASK \
if (s->rNToGo == 0) { \
s->rNToGo = BZ2_rNums[s->rTPos]; \
s->rTPos++; \
if (s->rTPos == 512) s->rTPos = 0; \
} \
s->rNToGo--;
/*-- Stuff for doing CRCs. --*/
extern UInt32 BZ2_crc32Table[256];
#define BZ_INITIALISE_CRC(crcVar) \
{ \
crcVar = 0xffffffffL; \
}
#define BZ_FINALISE_CRC(crcVar) \
{ \
crcVar = ~(crcVar); \
}
#define BZ_UPDATE_CRC(crcVar,cha) \
{ \
crcVar = (crcVar << 8) ^ \
BZ2_crc32Table[(crcVar >> 24) ^ \
((UChar)cha)]; \
}
/*-- States and modes for compression. --*/
#define BZ_M_IDLE 1
#define BZ_M_RUNNING 2
#define BZ_M_FLUSHING 3
#define BZ_M_FINISHING 4
#define BZ_S_OUTPUT 1
#define BZ_S_INPUT 2
#define BZ_N_RADIX 2
#define BZ_N_QSORT 12
#define BZ_N_SHELL 18
#define BZ_N_OVERSHOOT (BZ_N_RADIX + BZ_N_QSORT + BZ_N_SHELL + 2)
/*-- Structure holding all the compression-side stuff. --*/
typedef
struct {
/* pointer back to the struct bz_stream */
bz_stream* strm;
/* mode this stream is in, and whether inputting */
/* or outputting data */
Int32 mode;
Int32 state;
/* remembers avail_in when flush/finish requested */
UInt32 avail_in_expect;
/* for doing the block sorting */
UInt32* arr1;
UInt32* arr2;
UInt32* ftab;
Int32 origPtr;
/* aliases for arr1 and arr2 */
UInt32* ptr;
UChar* block;
UInt16* mtfv;
UChar* zbits;
/* for deciding when to use the fallback sorting algorithm */
Int32 workFactor;
/* run-length-encoding of the input */
UInt32 state_in_ch;
Int32 state_in_len;
BZ_RAND_DECLS;
/* input and output limits and current posns */
Int32 nblock;
Int32 nblockMAX;
Int32 numZ;
Int32 state_out_pos;
/* map of bytes used in block */
Int32 nInUse;
Bool inUse[256];
UChar unseqToSeq[256];
/* the buffer for bit stream creation */
UInt32 bsBuff;
Int32 bsLive;
/* block and combined CRCs */
UInt32 blockCRC;
UInt32 combinedCRC;
/* misc administratium */
Int32 verbosity;
Int32 blockNo;
Int32 blockSize100k;
/* stuff for coding the MTF values */
Int32 nMTF;
Int32 mtfFreq [BZ_MAX_ALPHA_SIZE];
UChar selector [BZ_MAX_SELECTORS];
UChar selectorMtf[BZ_MAX_SELECTORS];
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 code [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 rfreq [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
/* second dimension: only 3 needed; 4 makes index calculations faster */
UInt32 len_pack[BZ_MAX_ALPHA_SIZE][4];
}
EState;
/*-- externs for compression. --*/
extern void
BZ2_blockSort ( EState* );
extern void
BZ2_compressBlock ( EState*, Bool );
extern void
BZ2_bsInitWrite ( EState* );
extern void
BZ2_hbAssignCodes ( Int32*, UChar*, Int32, Int32, Int32 );
extern void
BZ2_hbMakeCodeLengths ( UChar*, Int32*, Int32, Int32 );
/*-- states for decompression. --*/
#define BZ_X_IDLE 1
#define BZ_X_OUTPUT 2
#define BZ_X_MAGIC_1 10
#define BZ_X_MAGIC_2 11
#define BZ_X_MAGIC_3 12
#define BZ_X_MAGIC_4 13
#define BZ_X_BLKHDR_1 14
#define BZ_X_BLKHDR_2 15
#define BZ_X_BLKHDR_3 16
#define BZ_X_BLKHDR_4 17
#define BZ_X_BLKHDR_5 18
#define BZ_X_BLKHDR_6 19
#define BZ_X_BCRC_1 20
#define BZ_X_BCRC_2 21
#define BZ_X_BCRC_3 22
#define BZ_X_BCRC_4 23
#define BZ_X_RANDBIT 24
#define BZ_X_ORIGPTR_1 25
#define BZ_X_ORIGPTR_2 26
#define BZ_X_ORIGPTR_3 27
#define BZ_X_MAPPING_1 28
#define BZ_X_MAPPING_2 29
#define BZ_X_SELECTOR_1 30
#define BZ_X_SELECTOR_2 31
#define BZ_X_SELECTOR_3 32
#define BZ_X_CODING_1 33
#define BZ_X_CODING_2 34
#define BZ_X_CODING_3 35
#define BZ_X_MTF_1 36
#define BZ_X_MTF_2 37
#define BZ_X_MTF_3 38
#define BZ_X_MTF_4 39
#define BZ_X_MTF_5 40
#define BZ_X_MTF_6 41
#define BZ_X_ENDHDR_2 42
#define BZ_X_ENDHDR_3 43
#define BZ_X_ENDHDR_4 44
#define BZ_X_ENDHDR_5 45
#define BZ_X_ENDHDR_6 46
#define BZ_X_CCRC_1 47
#define BZ_X_CCRC_2 48
#define BZ_X_CCRC_3 49
#define BZ_X_CCRC_4 50
/*-- Constants for the fast MTF decoder. --*/
#define MTFA_SIZE 4096
#define MTFL_SIZE 16
/*-- Structure holding all the decompression-side stuff. --*/
typedef
struct {
/* pointer back to the struct bz_stream */
bz_stream* strm;
/* state indicator for this stream */
Int32 state;
/* for doing the final run-length decoding */
UChar state_out_ch;
Int32 state_out_len;
Bool blockRandomised;
BZ_RAND_DECLS;
/* the buffer for bit stream reading */
UInt32 bsBuff;
Int32 bsLive;
/* misc administratium */
Int32 blockSize100k;
Bool smallDecompress;
Int32 currBlockNo;
Int32 verbosity;
/* for undoing the Burrows-Wheeler transform */
Int32 origPtr;
UInt32 tPos;
Int32 k0;
Int32 unzftab[256];
Int32 nblock_used;
Int32 cftab[257];
Int32 cftabCopy[257];
/* for undoing the Burrows-Wheeler transform (FAST) */
UInt32 *tt;
/* for undoing the Burrows-Wheeler transform (SMALL) */
UInt16 *ll16;
UChar *ll4;
/* stored and calculated CRCs */
UInt32 storedBlockCRC;
UInt32 storedCombinedCRC;
UInt32 calculatedBlockCRC;
UInt32 calculatedCombinedCRC;
/* map of bytes used in block */
Int32 nInUse;
Bool inUse[256];
Bool inUse16[16];
UChar seqToUnseq[256];
/* for decoding the MTF values */
UChar mtfa [MTFA_SIZE];
Int32 mtfbase[256 / MTFL_SIZE];
UChar selector [BZ_MAX_SELECTORS];
UChar selectorMtf[BZ_MAX_SELECTORS];
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 limit [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 base [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 perm [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 minLens[BZ_N_GROUPS];
/* save area for scalars in the main decompress code */
Int32 save_i;
Int32 save_j;
Int32 save_t;
Int32 save_alphaSize;
Int32 save_nGroups;
Int32 save_nSelectors;
Int32 save_EOB;
Int32 save_groupNo;
Int32 save_groupPos;
Int32 save_nextSym;
Int32 save_nblockMAX;
Int32 save_nblock;
Int32 save_es;
Int32 save_N;
Int32 save_curr;
Int32 save_zt;
Int32 save_zn;
Int32 save_zvec;
Int32 save_zj;
Int32 save_gSel;
Int32 save_gMinlen;
Int32* save_gLimit;
Int32* save_gBase;
Int32* save_gPerm;
}
DState;
/*-- Macros for decompression. --*/
#define BZ_GET_FAST(cccc) \
s->tPos = s->tt[s->tPos]; \
cccc = (UChar)(s->tPos & 0xff); \
s->tPos >>= 8;
#define BZ_GET_FAST_C(cccc) \
c_tPos = c_tt[c_tPos]; \
cccc = (UChar)(c_tPos & 0xff); \
c_tPos >>= 8;
#define SET_LL4(i,n) \
{ if (((i) & 0x1) == 0) \
s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0xf0) | (n); else \
s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0x0f) | ((n) << 4); \
}
#define GET_LL4(i) \
((((UInt32)(s->ll4[(i) >> 1])) >> (((i) << 2) & 0x4)) & 0xF)
#define SET_LL(i,n) \
{ s->ll16[i] = (UInt16)(n & 0x0000ffff); \
SET_LL4(i, n >> 16); \
}
#define GET_LL(i) \
(((UInt32)s->ll16[i]) | (GET_LL4(i) << 16))
#define BZ_GET_SMALL(cccc) \
cccc = BZ2_indexIntoF ( s->tPos, s->cftab ); \
s->tPos = GET_LL(s->tPos);
/*-- externs for decompression. --*/
extern Int32
BZ2_indexIntoF ( Int32, Int32* );
extern Int32
BZ2_decompress ( DState* );
extern void
BZ2_hbCreateDecodeTables ( Int32*, Int32*, Int32*, UChar*,
Int32, Int32, Int32 );
#endif
/*-- BZ_NO_STDIO seems to make NULL disappear on some platforms. --*/
#ifdef BZ_NO_STDIO
#ifndef NULL
#define NULL 0
#endif
#endif
/*-------------------------------------------------------------*/
/*--- end bzlib_private.h ---*/
/*-------------------------------------------------------------*/

61
contrib/vmap_extractor_v2/stormlib/bzip2/bzmore
View file

@ -1,61 +0,0 @@
#!/bin/sh
# Bzmore wrapped for bzip2,
# adapted from zmore by Philippe Troin <phil@fifi.org> for Debian GNU/Linux.
PATH="/usr/bin:$PATH"; export PATH
prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
*less) more=less ;;
*) more=more ;;
esac
if test "`echo -n a`" = "-n a"; then
# looks like a SysV system:
n1=''; n2='\c'
else
n1='-n'; n2=''
fi
oldtty=`stty -g 2>/dev/null`
if stty -cbreak 2>/dev/null; then
cb='cbreak'; ncb='-cbreak'
else
# 'stty min 1' resets eof to ^a on both SunOS and SysV!
cb='min 1 -icanon'; ncb='icanon eof ^d'
fi
if test $? -eq 0 -a -n "$oldtty"; then
trap 'stty $oldtty 2>/dev/null; exit' 0 2 3 5 10 13 15
else
trap 'stty $ncb echo 2>/dev/null; exit' 0 2 3 5 10 13 15
fi
if test $# = 0; then
if test -t 0; then
echo usage: $prog files...
else
bzip2 -cdfq | eval $more
fi
else
FIRST=1
for FILE
do
if test $FIRST -eq 0; then
echo $n1 "--More--(Next file: $FILE)$n2"
stty $cb -echo 2>/dev/null
ANS=`dd bs=1 count=1 2>/dev/null`
stty $ncb echo 2>/dev/null
echo " "
if test "$ANS" = 'e' -o "$ANS" = 'q'; then
exit
fi
fi
if test "$ANS" != 's'; then
echo "------> $FILE <------"
bzip2 -cdfq "$FILE" | eval $more
fi
if test -t; then
FIRST=0
fi
done
fi

152
contrib/vmap_extractor_v2/stormlib/bzip2/bzmore.1
View file

@ -1,152 +0,0 @@
.\"Shamelessly copied from zmore.1 by Philippe Troin <phil@fifi.org>
.\"for Debian GNU/Linux
.TH BZMORE 1
.SH NAME
bzmore, bzless \- file perusal filter for crt viewing of bzip2 compressed text
.SH SYNOPSIS
.B bzmore
[ name ... ]
.br
.B bzless
[ name ... ]
.SH NOTE
In the following description,
.I bzless
and
.I less
can be used interchangeably with
.I bzmore
and
.I more.
.SH DESCRIPTION
.I Bzmore
is a filter which allows examination of compressed or plain text files
one screenful at a time on a soft-copy terminal.
.I bzmore
works on files compressed with
.I bzip2
and also on uncompressed files.
If a file does not exist,
.I bzmore
looks for a file of the same name with the addition of a .bz2 suffix.
.PP
.I Bzmore
normally pauses after each screenful, printing --More--
at the bottom of the screen.
If the user then types a carriage return, one more line is displayed.
If the user hits a space,
another screenful is displayed. Other possibilities are enumerated later.
.PP
.I Bzmore
looks in the file
.I /etc/termcap
to determine terminal characteristics,
and to determine the default window size.
On a terminal capable of displaying 24 lines,
the default window size is 22 lines.
Other sequences which may be typed when
.I bzmore
pauses, and their effects, are as follows (\fIi\fP is an optional integer
argument, defaulting to 1) :
.PP
.IP \fIi\|\fP<space>
display
.I i
more lines, (or another screenful if no argument is given)
.PP
.IP ^D
display 11 more lines (a ``scroll'').
If
.I i
is given, then the scroll size is set to \fIi\|\fP.
.PP
.IP d
same as ^D (control-D)
.PP
.IP \fIi\|\fPz
same as typing a space except that \fIi\|\fP, if present, becomes the new
window size. Note that the window size reverts back to the default at the
end of the current file.
.PP
.IP \fIi\|\fPs
skip \fIi\|\fP lines and print a screenful of lines
.PP
.IP \fIi\|\fPf
skip \fIi\fP screenfuls and print a screenful of lines
.PP
.IP "q or Q"
quit reading the current file; go on to the next (if any)
.PP
.IP "e or q"
When the prompt --More--(Next file:
.IR file )
is printed, this command causes bzmore to exit.
.PP
.IP s
When the prompt --More--(Next file:
.IR file )
is printed, this command causes bzmore to skip the next file and continue.
.PP
.IP =
Display the current line number.
.PP
.IP \fIi\|\fP/expr
search for the \fIi\|\fP-th occurrence of the regular expression \fIexpr.\fP
If the pattern is not found,
.I bzmore
goes on to the next file (if any).
Otherwise, a screenful is displayed, starting two lines before the place
where the expression was found.
The user's erase and kill characters may be used to edit the regular
expression.
Erasing back past the first column cancels the search command.
.PP
.IP \fIi\|\fPn
search for the \fIi\|\fP-th occurrence of the last regular expression entered.
.PP
.IP !command
invoke a shell with \fIcommand\|\fP.
The character `!' in "command" are replaced with the
previous shell command. The sequence "\\!" is replaced by "!".
.PP
.IP ":q or :Q"
quit reading the current file; go on to the next (if any)
(same as q or Q).
.PP
.IP .
(dot) repeat the previous command.
.PP
The commands take effect immediately, i.e., it is not necessary to
type a carriage return.
Up to the time when the command character itself is given,
the user may hit the line kill character to cancel the numerical
argument being formed.
In addition, the user may hit the erase character to redisplay the
--More-- message.
.PP
At any time when output is being sent to the terminal, the user can
hit the quit key (normally control\-\\).
.I Bzmore
will stop sending output, and will display the usual --More--
prompt.
The user may then enter one of the above commands in the normal manner.
Unfortunately, some output is lost when this is done, due to the
fact that any characters waiting in the terminal's output queue
are flushed when the quit signal occurs.
.PP
The terminal is set to
.I noecho
mode by this program so that the output can be continuous.
What you type will thus not show on your terminal, except for the / and !
commands.
.PP
If the standard output is not a teletype, then
.I bzmore
acts just like
.I bzcat,
except that a header is printed before each file.
.SH FILES
.DT
/etc/termcap Terminal data base
.SH "SEE ALSO"
more(1), less(1), bzip2(1), bzdiff(1), bzgrep(1)

715
contrib/vmap_extractor_v2/stormlib/bzip2/compress.c
View file

@ -1,715 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Compression machinery (not incl block sorting) ---*/
/*--- compress.c ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
/*--
CHANGES
~~~~~~~
0.9.0 -- original version.
0.9.0a/b -- no changes in this file.
0.9.0c
* changed setting of nGroups in sendMTFValues() so as to
do a bit better on small files
--*/
#include "bzlib_private.h"
/*---------------------------------------------------*/
/*--- Bit stream I/O ---*/
/*---------------------------------------------------*/
/*---------------------------------------------------*/
void BZ2_bsInitWrite ( EState* s )
{
s->bsLive = 0;
s->bsBuff = 0;
}
/*---------------------------------------------------*/
static
void bsFinishWrite ( EState* s )
{
while (s->bsLive > 0) {
s->zbits[s->numZ] = (UChar)(s->bsBuff >> 24);
s->numZ++;
s->bsBuff <<= 8;
s->bsLive -= 8;
}
}
/*---------------------------------------------------*/
#define bsNEEDW(nz) \
{ \
while (s->bsLive >= 8) { \
s->zbits[s->numZ] \
= (UChar)(s->bsBuff >> 24); \
s->numZ++; \
s->bsBuff <<= 8; \
s->bsLive -= 8; \
} \
}
/*---------------------------------------------------*/
static
__inline__
void bsW ( EState* s, Int32 n, UInt32 v )
{
bsNEEDW ( n );
s->bsBuff |= (v << (32 - s->bsLive - n));
s->bsLive += n;
}
/*---------------------------------------------------*/
static
void bsPutUInt32 ( EState* s, UInt32 u )
{
bsW ( s, 8, (u >> 24) & 0xffL );
bsW ( s, 8, (u >> 16) & 0xffL );
bsW ( s, 8, (u >> 8) & 0xffL );
bsW ( s, 8, u & 0xffL );
}
/*---------------------------------------------------*/
static
void bsPutUChar ( EState* s, UChar c )
{
bsW( s, 8, (UInt32)c );
}
/*---------------------------------------------------*/
/*--- The back end proper ---*/
/*---------------------------------------------------*/
/*---------------------------------------------------*/
static
void makeMaps_e ( EState* s )
{
Int32 i;
s->nInUse = 0;
for (i = 0; i < 256; i++)
if (s->inUse[i]) {
s->unseqToSeq[i] = s->nInUse;
s->nInUse++;
}
}
/*---------------------------------------------------*/
static
void generateMTFValues ( EState* s )
{
UChar yy[256];
Int32 i, j;
Int32 zPend;
Int32 wr;
Int32 EOB;
/*
After sorting (eg, here),
s->arr1 [ 0 .. s->nblock-1 ] holds sorted order,
and
((UChar*)s->arr2) [ 0 .. s->nblock-1 ]
holds the original block data.
The first thing to do is generate the MTF values,
and put them in
((UInt16*)s->arr1) [ 0 .. s->nblock-1 ].
Because there are strictly fewer or equal MTF values
than block values, ptr values in this area are overwritten
with MTF values only when they are no longer needed.
The final compressed bitstream is generated into the
area starting at
(UChar*) (&((UChar*)s->arr2)[s->nblock])
These storage aliases are set up in bzCompressInit(),
except for the last one, which is arranged in
compressBlock().
*/
UInt32* ptr = s->ptr;
UChar* block = s->block;
UInt16* mtfv = s->mtfv;
makeMaps_e ( s );
EOB = s->nInUse+1;
for (i = 0; i <= EOB; i++) s->mtfFreq[i] = 0;
wr = 0;
zPend = 0;
for (i = 0; i < s->nInUse; i++) yy[i] = (UChar) i;
for (i = 0; i < s->nblock; i++) {
UChar ll_i;
AssertD ( wr <= i, "generateMTFValues(1)" );
j = ptr[i]-1; if (j < 0) j += s->nblock;
ll_i = s->unseqToSeq[block[j]];
AssertD ( ll_i < s->nInUse, "generateMTFValues(2a)" );
if (yy[0] == ll_i) {
zPend++;
} else {
if (zPend > 0) {
zPend--;
while (True) {
if (zPend & 1) {
mtfv[wr] = BZ_RUNB; wr++;
s->mtfFreq[BZ_RUNB]++;
} else {
mtfv[wr] = BZ_RUNA; wr++;
s->mtfFreq[BZ_RUNA]++;
}
if (zPend < 2) break;
zPend = (zPend - 2) / 2;
};
zPend = 0;
}
{
register UChar rtmp;
register UChar* ryy_j;
register UChar rll_i;
rtmp = yy[1];
yy[1] = yy[0];
ryy_j = &(yy[1]);
rll_i = ll_i;
while ( rll_i != rtmp ) {
register UChar rtmp2;
ryy_j++;
rtmp2 = rtmp;
rtmp = *ryy_j;
*ryy_j = rtmp2;
};
yy[0] = rtmp;
j = ryy_j - &(yy[0]);
mtfv[wr] = j+1; wr++; s->mtfFreq[j+1]++;
}
}
}
if (zPend > 0) {
zPend--;
while (True) {
if (zPend & 1) {
mtfv[wr] = BZ_RUNB; wr++;
s->mtfFreq[BZ_RUNB]++;
} else {
mtfv[wr] = BZ_RUNA; wr++;
s->mtfFreq[BZ_RUNA]++;
}
if (zPend < 2) break;
zPend = (zPend - 2) / 2;
};
zPend = 0;
}
mtfv[wr] = EOB; wr++; s->mtfFreq[EOB]++;
s->nMTF = wr;
}
/*---------------------------------------------------*/
#define BZ_LESSER_ICOST 0
#define BZ_GREATER_ICOST 15
static
void sendMTFValues ( EState* s )
{
Int32 v, t, i, j, gs, ge, totc, bt, bc, iter;
Int32 nSelectors, alphaSize, minLen, maxLen, selCtr;
Int32 nGroups, nBytes;
/*--
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
is a global since the decoder also needs it.
Int32 code[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 rfreq[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
are also globals only used in this proc.
Made global to keep stack frame size small.
--*/
UInt16 cost[BZ_N_GROUPS];
Int32 fave[BZ_N_GROUPS];
UInt16* mtfv = s->mtfv;
if (s->verbosity >= 3)
VPrintf3( " %d in block, %d after MTF & 1-2 coding, "
"%d+2 syms in use\n",
s->nblock, s->nMTF, s->nInUse );
alphaSize = s->nInUse+2;
for (t = 0; t < BZ_N_GROUPS; t++)
for (v = 0; v < alphaSize; v++)
s->len[t][v] = BZ_GREATER_ICOST;
/*--- Decide how many coding tables to use ---*/
AssertH ( s->nMTF > 0, 3001 );
if (s->nMTF < 200) nGroups = 2; else
if (s->nMTF < 600) nGroups = 3; else
if (s->nMTF < 1200) nGroups = 4; else
if (s->nMTF < 2400) nGroups = 5; else
nGroups = 6;
/*--- Generate an initial set of coding tables ---*/
{
Int32 nPart, remF, tFreq, aFreq;
nPart = nGroups;
remF = s->nMTF;
gs = 0;
while (nPart > 0) {
tFreq = remF / nPart;
ge = gs-1;
aFreq = 0;
while (aFreq < tFreq && ge < alphaSize-1) {
ge++;
aFreq += s->mtfFreq[ge];
}
if (ge > gs
&& nPart != nGroups && nPart != 1
&& ((nGroups-nPart) % 2 == 1)) {
aFreq -= s->mtfFreq[ge];
ge--;
}
if (s->verbosity >= 3)
VPrintf5( " initial group %d, [%d .. %d], "
"has %d syms (%4.1f%%)\n",
nPart, gs, ge, aFreq,
(100.0 * (float)aFreq) / (float)(s->nMTF) );
for (v = 0; v < alphaSize; v++)
if (v >= gs && v <= ge)
s->len[nPart-1][v] = BZ_LESSER_ICOST; else
s->len[nPart-1][v] = BZ_GREATER_ICOST;
nPart--;
gs = ge+1;
remF -= aFreq;
}
}
/*---
Iterate up to BZ_N_ITERS times to improve the tables.
---*/
for (iter = 0; iter < BZ_N_ITERS; iter++) {
for (t = 0; t < nGroups; t++) fave[t] = 0;
for (t = 0; t < nGroups; t++)
for (v = 0; v < alphaSize; v++)
s->rfreq[t][v] = 0;
/*---
Set up an auxiliary length table which is used to fast-track
the common case (nGroups == 6).
---*/
if (nGroups == 6) {
for (v = 0; v < alphaSize; v++) {
s->len_pack[v][0] = (s->len[1][v] << 16) | s->len[0][v];
s->len_pack[v][1] = (s->len[3][v] << 16) | s->len[2][v];
s->len_pack[v][2] = (s->len[5][v] << 16) | s->len[4][v];
}
}
nSelectors = 0;
totc = 0;
gs = 0;
while (True) {
/*--- Set group start & end marks. --*/
if (gs >= s->nMTF) break;
ge = gs + BZ_G_SIZE - 1;
if (ge >= s->nMTF) ge = s->nMTF-1;
/*--
Calculate the cost of this group as coded
by each of the coding tables.
--*/
for (t = 0; t < nGroups; t++) cost[t] = 0;
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
register UInt32 cost01, cost23, cost45;
register UInt16 icv;
cost01 = cost23 = cost45 = 0;
# define BZ_ITER(nn) \
icv = mtfv[gs+(nn)]; \
cost01 += s->len_pack[icv][0]; \
cost23 += s->len_pack[icv][1]; \
cost45 += s->len_pack[icv][2]; \
BZ_ITER(0); BZ_ITER(1); BZ_ITER(2); BZ_ITER(3); BZ_ITER(4);
BZ_ITER(5); BZ_ITER(6); BZ_ITER(7); BZ_ITER(8); BZ_ITER(9);
BZ_ITER(10); BZ_ITER(11); BZ_ITER(12); BZ_ITER(13); BZ_ITER(14);
BZ_ITER(15); BZ_ITER(16); BZ_ITER(17); BZ_ITER(18); BZ_ITER(19);
BZ_ITER(20); BZ_ITER(21); BZ_ITER(22); BZ_ITER(23); BZ_ITER(24);
BZ_ITER(25); BZ_ITER(26); BZ_ITER(27); BZ_ITER(28); BZ_ITER(29);
BZ_ITER(30); BZ_ITER(31); BZ_ITER(32); BZ_ITER(33); BZ_ITER(34);
BZ_ITER(35); BZ_ITER(36); BZ_ITER(37); BZ_ITER(38); BZ_ITER(39);
BZ_ITER(40); BZ_ITER(41); BZ_ITER(42); BZ_ITER(43); BZ_ITER(44);
BZ_ITER(45); BZ_ITER(46); BZ_ITER(47); BZ_ITER(48); BZ_ITER(49);
# undef BZ_ITER
cost[0] = cost01 & 0xffff; cost[1] = cost01 >> 16;
cost[2] = cost23 & 0xffff; cost[3] = cost23 >> 16;
cost[4] = cost45 & 0xffff; cost[5] = cost45 >> 16;
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++) {
UInt16 icv = mtfv[i];
for (t = 0; t < nGroups; t++) cost[t] += s->len[t][icv];
}
}
/*--
Find the coding table which is best for this group,
and record its identity in the selector table.
--*/
bc = 999999999; bt = -1;
for (t = 0; t < nGroups; t++)
if (cost[t] < bc) { bc = cost[t]; bt = t; };
totc += bc;
fave[bt]++;
s->selector[nSelectors] = bt;
nSelectors++;
/*--
Increment the symbol frequencies for the selected table.
--*/
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
# define BZ_ITUR(nn) s->rfreq[bt][ mtfv[gs+(nn)] ]++
BZ_ITUR(0); BZ_ITUR(1); BZ_ITUR(2); BZ_ITUR(3); BZ_ITUR(4);
BZ_ITUR(5); BZ_ITUR(6); BZ_ITUR(7); BZ_ITUR(8); BZ_ITUR(9);
BZ_ITUR(10); BZ_ITUR(11); BZ_ITUR(12); BZ_ITUR(13); BZ_ITUR(14);
BZ_ITUR(15); BZ_ITUR(16); BZ_ITUR(17); BZ_ITUR(18); BZ_ITUR(19);
BZ_ITUR(20); BZ_ITUR(21); BZ_ITUR(22); BZ_ITUR(23); BZ_ITUR(24);
BZ_ITUR(25); BZ_ITUR(26); BZ_ITUR(27); BZ_ITUR(28); BZ_ITUR(29);
BZ_ITUR(30); BZ_ITUR(31); BZ_ITUR(32); BZ_ITUR(33); BZ_ITUR(34);
BZ_ITUR(35); BZ_ITUR(36); BZ_ITUR(37); BZ_ITUR(38); BZ_ITUR(39);
BZ_ITUR(40); BZ_ITUR(41); BZ_ITUR(42); BZ_ITUR(43); BZ_ITUR(44);
BZ_ITUR(45); BZ_ITUR(46); BZ_ITUR(47); BZ_ITUR(48); BZ_ITUR(49);
# undef BZ_ITUR
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++)
s->rfreq[bt][ mtfv[i] ]++;
}
gs = ge+1;
}
if (s->verbosity >= 3) {
VPrintf2 ( " pass %d: size is %d, grp uses are ",
iter+1, totc/8 );
for (t = 0; t < nGroups; t++)
VPrintf1 ( "%d ", fave[t] );
VPrintf0 ( "\n" );
}
/*--
Recompute the tables based on the accumulated frequencies.
--*/
/* maxLen was changed from 20 to 17 in bzip2-1.0.3. See
comment in huffman.c for details. */
for (t = 0; t < nGroups; t++)
BZ2_hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]),
alphaSize, 17 /*20*/ );
}
AssertH( nGroups < 8, 3002 );
AssertH( nSelectors < 32768 &&
nSelectors <= (2 + (900000 / BZ_G_SIZE)),
3003 );
/*--- Compute MTF values for the selectors. ---*/
{
UChar pos[BZ_N_GROUPS], ll_i, tmp2, tmp;
for (i = 0; i < nGroups; i++) pos[i] = i;
for (i = 0; i < nSelectors; i++) {
ll_i = s->selector[i];
j = 0;
tmp = pos[j];
while ( ll_i != tmp ) {
j++;
tmp2 = tmp;
tmp = pos[j];
pos[j] = tmp2;
};
pos[0] = tmp;
s->selectorMtf[i] = j;
}
};
/*--- Assign actual codes for the tables. --*/
for (t = 0; t < nGroups; t++) {
minLen = 32;
maxLen = 0;
for (i = 0; i < alphaSize; i++) {
if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
if (s->len[t][i] < minLen) minLen = s->len[t][i];
}
AssertH ( !(maxLen > 17 /*20*/ ), 3004 );
AssertH ( !(minLen < 1), 3005 );
BZ2_hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]),
minLen, maxLen, alphaSize );
}
/*--- Transmit the mapping table. ---*/
{
Bool inUse16[16];
for (i = 0; i < 16; i++) {
inUse16[i] = False;
for (j = 0; j < 16; j++)
if (s->inUse[i * 16 + j]) inUse16[i] = True;
}
nBytes = s->numZ;
for (i = 0; i < 16; i++)
if (inUse16[i]) bsW(s,1,1); else bsW(s,1,0);
for (i = 0; i < 16; i++)
if (inUse16[i])
for (j = 0; j < 16; j++) {
if (s->inUse[i * 16 + j]) bsW(s,1,1); else bsW(s,1,0);
}
if (s->verbosity >= 3)
VPrintf1( " bytes: mapping %d, ", s->numZ-nBytes );
}
/*--- Now the selectors. ---*/
nBytes = s->numZ;
bsW ( s, 3, nGroups );
bsW ( s, 15, nSelectors );
for (i = 0; i < nSelectors; i++) {
for (j = 0; j < s->selectorMtf[i]; j++) bsW(s,1,1);
bsW(s,1,0);
}
if (s->verbosity >= 3)
VPrintf1( "selectors %d, ", s->numZ-nBytes );
/*--- Now the coding tables. ---*/
nBytes = s->numZ;
for (t = 0; t < nGroups; t++) {
Int32 curr = s->len[t][0];
bsW ( s, 5, curr );
for (i = 0; i < alphaSize; i++) {
while (curr < s->len[t][i]) { bsW(s,2,2); curr++; /* 10 */ };
while (curr > s->len[t][i]) { bsW(s,2,3); curr--; /* 11 */ };
bsW ( s, 1, 0 );
}
}
if (s->verbosity >= 3)
VPrintf1 ( "code lengths %d, ", s->numZ-nBytes );
/*--- And finally, the block data proper ---*/
nBytes = s->numZ;
selCtr = 0;
gs = 0;
while (True) {
if (gs >= s->nMTF) break;
ge = gs + BZ_G_SIZE - 1;
if (ge >= s->nMTF) ge = s->nMTF-1;
AssertH ( s->selector[selCtr] < nGroups, 3006 );
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
UInt16 mtfv_i;
UChar* s_len_sel_selCtr
= &(s->len[s->selector[selCtr]][0]);
Int32* s_code_sel_selCtr
= &(s->code[s->selector[selCtr]][0]);
# define BZ_ITAH(nn) \
mtfv_i = mtfv[gs+(nn)]; \
bsW ( s, \
s_len_sel_selCtr[mtfv_i], \
s_code_sel_selCtr[mtfv_i] )
BZ_ITAH(0); BZ_ITAH(1); BZ_ITAH(2); BZ_ITAH(3); BZ_ITAH(4);
BZ_ITAH(5); BZ_ITAH(6); BZ_ITAH(7); BZ_ITAH(8); BZ_ITAH(9);
BZ_ITAH(10); BZ_ITAH(11); BZ_ITAH(12); BZ_ITAH(13); BZ_ITAH(14);
BZ_ITAH(15); BZ_ITAH(16); BZ_ITAH(17); BZ_ITAH(18); BZ_ITAH(19);
BZ_ITAH(20); BZ_ITAH(21); BZ_ITAH(22); BZ_ITAH(23); BZ_ITAH(24);
BZ_ITAH(25); BZ_ITAH(26); BZ_ITAH(27); BZ_ITAH(28); BZ_ITAH(29);
BZ_ITAH(30); BZ_ITAH(31); BZ_ITAH(32); BZ_ITAH(33); BZ_ITAH(34);
BZ_ITAH(35); BZ_ITAH(36); BZ_ITAH(37); BZ_ITAH(38); BZ_ITAH(39);
BZ_ITAH(40); BZ_ITAH(41); BZ_ITAH(42); BZ_ITAH(43); BZ_ITAH(44);
BZ_ITAH(45); BZ_ITAH(46); BZ_ITAH(47); BZ_ITAH(48); BZ_ITAH(49);
# undef BZ_ITAH
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++) {
bsW ( s,
s->len [s->selector[selCtr]] [mtfv[i]],
s->code [s->selector[selCtr]] [mtfv[i]] );
}
}
gs = ge+1;
selCtr++;
}
AssertH( selCtr == nSelectors, 3007 );
if (s->verbosity >= 3)
VPrintf1( "codes %d\n", s->numZ-nBytes );
}
/*---------------------------------------------------*/
void BZ2_compressBlock ( EState* s, Bool is_last_block )
{
if (s->nblock > 0) {
BZ_FINALISE_CRC ( s->blockCRC );
s->combinedCRC = (s->combinedCRC << 1) | (s->combinedCRC >> 31);
s->combinedCRC ^= s->blockCRC;
if (s->blockNo > 1) s->numZ = 0;
if (s->verbosity >= 2)
VPrintf4( " block %d: crc = 0x%08x, "
"combined CRC = 0x%08x, size = %d\n",
s->blockNo, s->blockCRC, s->combinedCRC, s->nblock );
BZ2_blockSort ( s );
}
s->zbits = (UChar*) (&((UChar*)s->arr2)[s->nblock]);
/*-- If this is the first block, create the stream header. --*/
if (s->blockNo == 1) {
BZ2_bsInitWrite ( s );
bsPutUChar ( s, BZ_HDR_B );
bsPutUChar ( s, BZ_HDR_Z );
bsPutUChar ( s, BZ_HDR_h );
bsPutUChar ( s, (UChar)(BZ_HDR_0 + s->blockSize100k) );
}
if (s->nblock > 0) {
bsPutUChar ( s, 0x31 ); bsPutUChar ( s, 0x41 );
bsPutUChar ( s, 0x59 ); bsPutUChar ( s, 0x26 );
bsPutUChar ( s, 0x53 ); bsPutUChar ( s, 0x59 );
/*-- Now the block's CRC, so it is in a known place. --*/
bsPutUInt32 ( s, s->blockCRC );
/*--
Now a single bit indicating (non-)randomisation.
As of version 0.9.5, we use a better sorting algorithm
which makes randomisation unnecessary. So always set
the randomised bit to 'no'. Of course, the decoder
still needs to be able to handle randomised blocks
so as to maintain backwards compatibility with
older versions of bzip2.
--*/
bsW(s,1,0);
bsW ( s, 24, s->origPtr );
generateMTFValues ( s );
sendMTFValues ( s );
}
/*-- If this is the last block, add the stream trailer. --*/
if (is_last_block) {
bsPutUChar ( s, 0x17 ); bsPutUChar ( s, 0x72 );
bsPutUChar ( s, 0x45 ); bsPutUChar ( s, 0x38 );
bsPutUChar ( s, 0x50 ); bsPutUChar ( s, 0x90 );
bsPutUInt32 ( s, s->combinedCRC );
if (s->verbosity >= 2)
VPrintf1( " final combined CRC = 0x%08x\n ", s->combinedCRC );
bsFinishWrite ( s );
}
}
/*-------------------------------------------------------------*/
/*--- end compress.c ---*/
/*-------------------------------------------------------------*/

144
contrib/vmap_extractor_v2/stormlib/bzip2/crctable.c
View file

@ -1,144 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Table for doing CRCs ---*/
/*--- crctable.c ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
#include "bzlib_private.h"
/*--
I think this is an implementation of the AUTODIN-II,
Ethernet & FDDI 32-bit CRC standard. Vaguely derived
from code by Rob Warnock, in Section 51 of the
comp.compression FAQ.
--*/
UInt32 BZ2_crc32Table[256] = {
/*-- Ugly, innit? --*/
0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L,
0x130476dcL, 0x17c56b6bL, 0x1a864db2L, 0x1e475005L,
0x2608edb8L, 0x22c9f00fL, 0x2f8ad6d6L, 0x2b4bcb61L,
0x350c9b64L, 0x31cd86d3L, 0x3c8ea00aL, 0x384fbdbdL,
0x4c11db70L, 0x48d0c6c7L, 0x4593e01eL, 0x4152fda9L,
0x5f15adacL, 0x5bd4b01bL, 0x569796c2L, 0x52568b75L,
0x6a1936c8L, 0x6ed82b7fL, 0x639b0da6L, 0x675a1011L,
0x791d4014L, 0x7ddc5da3L, 0x709f7b7aL, 0x745e66cdL,
0x9823b6e0L, 0x9ce2ab57L, 0x91a18d8eL, 0x95609039L,
0x8b27c03cL, 0x8fe6dd8bL, 0x82a5fb52L, 0x8664e6e5L,
0xbe2b5b58L, 0xbaea46efL, 0xb7a96036L, 0xb3687d81L,
0xad2f2d84L, 0xa9ee3033L, 0xa4ad16eaL, 0xa06c0b5dL,
0xd4326d90L, 0xd0f37027L, 0xddb056feL, 0xd9714b49L,
0xc7361b4cL, 0xc3f706fbL, 0xceb42022L, 0xca753d95L,
0xf23a8028L, 0xf6fb9d9fL, 0xfbb8bb46L, 0xff79a6f1L,
0xe13ef6f4L, 0xe5ffeb43L, 0xe8bccd9aL, 0xec7dd02dL,
0x34867077L, 0x30476dc0L, 0x3d044b19L, 0x39c556aeL,
0x278206abL, 0x23431b1cL, 0x2e003dc5L, 0x2ac12072L,
0x128e9dcfL, 0x164f8078L, 0x1b0ca6a1L, 0x1fcdbb16L,
0x018aeb13L, 0x054bf6a4L, 0x0808d07dL, 0x0cc9cdcaL,
0x7897ab07L, 0x7c56b6b0L, 0x71159069L, 0x75d48ddeL,
0x6b93dddbL, 0x6f52c06cL, 0x6211e6b5L, 0x66d0fb02L,
0x5e9f46bfL, 0x5a5e5b08L, 0x571d7dd1L, 0x53dc6066L,
0x4d9b3063L, 0x495a2dd4L, 0x44190b0dL, 0x40d816baL,
0xaca5c697L, 0xa864db20L, 0xa527fdf9L, 0xa1e6e04eL,
0xbfa1b04bL, 0xbb60adfcL, 0xb6238b25L, 0xb2e29692L,
0x8aad2b2fL, 0x8e6c3698L, 0x832f1041L, 0x87ee0df6L,
0x99a95df3L, 0x9d684044L, 0x902b669dL, 0x94ea7b2aL,
0xe0b41de7L, 0xe4750050L, 0xe9362689L, 0xedf73b3eL,
0xf3b06b3bL, 0xf771768cL, 0xfa325055L, 0xfef34de2L,
0xc6bcf05fL, 0xc27dede8L, 0xcf3ecb31L, 0xcbffd686L,
0xd5b88683L, 0xd1799b34L, 0xdc3abdedL, 0xd8fba05aL,
0x690ce0eeL, 0x6dcdfd59L, 0x608edb80L, 0x644fc637L,
0x7a089632L, 0x7ec98b85L, 0x738aad5cL, 0x774bb0ebL,
0x4f040d56L, 0x4bc510e1L, 0x46863638L, 0x42472b8fL,
0x5c007b8aL, 0x58c1663dL, 0x558240e4L, 0x51435d53L,
0x251d3b9eL, 0x21dc2629L, 0x2c9f00f0L, 0x285e1d47L,
0x36194d42L, 0x32d850f5L, 0x3f9b762cL, 0x3b5a6b9bL,
0x0315d626L, 0x07d4cb91L, 0x0a97ed48L, 0x0e56f0ffL,
0x1011a0faL, 0x14d0bd4dL, 0x19939b94L, 0x1d528623L,
0xf12f560eL, 0xf5ee4bb9L, 0xf8ad6d60L, 0xfc6c70d7L,
0xe22b20d2L, 0xe6ea3d65L, 0xeba91bbcL, 0xef68060bL,
0xd727bbb6L, 0xd3e6a601L, 0xdea580d8L, 0xda649d6fL,
0xc423cd6aL, 0xc0e2d0ddL, 0xcda1f604L, 0xc960ebb3L,
0xbd3e8d7eL, 0xb9ff90c9L, 0xb4bcb610L, 0xb07daba7L,
0xae3afba2L, 0xaafbe615L, 0xa7b8c0ccL, 0xa379dd7bL,
0x9b3660c6L, 0x9ff77d71L, 0x92b45ba8L, 0x9675461fL,
0x8832161aL, 0x8cf30badL, 0x81b02d74L, 0x857130c3L,
0x5d8a9099L, 0x594b8d2eL, 0x5408abf7L, 0x50c9b640L,
0x4e8ee645L, 0x4a4ffbf2L, 0x470cdd2bL, 0x43cdc09cL,
0x7b827d21L, 0x7f436096L, 0x7200464fL, 0x76c15bf8L,
0x68860bfdL, 0x6c47164aL, 0x61043093L, 0x65c52d24L,
0x119b4be9L, 0x155a565eL, 0x18197087L, 0x1cd86d30L,
0x029f3d35L, 0x065e2082L, 0x0b1d065bL, 0x0fdc1becL,
0x3793a651L, 0x3352bbe6L, 0x3e119d3fL, 0x3ad08088L,
0x2497d08dL, 0x2056cd3aL, 0x2d15ebe3L, 0x29d4f654L,
0xc5a92679L, 0xc1683bceL, 0xcc2b1d17L, 0xc8ea00a0L,
0xd6ad50a5L, 0xd26c4d12L, 0xdf2f6bcbL, 0xdbee767cL,
0xe3a1cbc1L, 0xe760d676L, 0xea23f0afL, 0xeee2ed18L,
0xf0a5bd1dL, 0xf464a0aaL, 0xf9278673L, 0xfde69bc4L,
0x89b8fd09L, 0x8d79e0beL, 0x803ac667L, 0x84fbdbd0L,
0x9abc8bd5L, 0x9e7d9662L, 0x933eb0bbL, 0x97ffad0cL,
0xafb010b1L, 0xab710d06L, 0xa6322bdfL, 0xa2f33668L,
0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L
};
/*-------------------------------------------------------------*/
/*--- end crctable.c ---*/
/*-------------------------------------------------------------*/

666
contrib/vmap_extractor_v2/stormlib/bzip2/decompress.c
View file

@ -1,666 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Decompression machinery ---*/
/*--- decompress.c ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
#include "bzlib_private.h"
/*---------------------------------------------------*/
static
void makeMaps_d ( DState* s )
{
Int32 i;
s->nInUse = 0;
for (i = 0; i < 256; i++)
if (s->inUse[i]) {
s->seqToUnseq[s->nInUse] = i;
s->nInUse++;
}
}
/*---------------------------------------------------*/
#define RETURN(rrr) \
{ retVal = rrr; goto save_state_and_return; };
#define GET_BITS(lll,vvv,nnn) \
case lll: s->state = lll; \
while (True) { \
if (s->bsLive >= nnn) { \
UInt32 v; \
v = (s->bsBuff >> \
(s->bsLive-nnn)) & ((1 << nnn)-1); \
s->bsLive -= nnn; \
vvv = v; \
break; \
} \
if (s->strm->avail_in == 0) RETURN(BZ_OK); \
s->bsBuff \
= (s->bsBuff << 8) | \
((UInt32) \
(*((UChar*)(s->strm->next_in)))); \
s->bsLive += 8; \
s->strm->next_in++; \
s->strm->avail_in--; \
s->strm->total_in_lo32++; \
if (s->strm->total_in_lo32 == 0) \
s->strm->total_in_hi32++; \
}
#define GET_UCHAR(lll,uuu) \
GET_BITS(lll,uuu,8)
#define GET_BIT(lll,uuu) \
GET_BITS(lll,uuu,1)
/*---------------------------------------------------*/
#define GET_MTF_VAL(label1,label2,lval) \
{ \
if (groupPos == 0) { \
groupNo++; \
if (groupNo >= nSelectors) \
RETURN(BZ_DATA_ERROR); \
groupPos = BZ_G_SIZE; \
gSel = s->selector[groupNo]; \
gMinlen = s->minLens[gSel]; \
gLimit = &(s->limit[gSel][0]); \
gPerm = &(s->perm[gSel][0]); \
gBase = &(s->base[gSel][0]); \
} \
groupPos--; \
zn = gMinlen; \
GET_BITS(label1, zvec, zn); \
while (1) { \
if (zn > 20 /* the longest code */) \
RETURN(BZ_DATA_ERROR); \
if (zvec <= gLimit[zn]) break; \
zn++; \
GET_BIT(label2, zj); \
zvec = (zvec << 1) | zj; \
}; \
if (zvec - gBase[zn] < 0 \
|| zvec - gBase[zn] >= BZ_MAX_ALPHA_SIZE) \
RETURN(BZ_DATA_ERROR); \
lval = gPerm[zvec - gBase[zn]]; \
}
/*---------------------------------------------------*/
Int32 BZ2_decompress ( DState* s )
{
UChar uc;
Int32 retVal;
Int32 minLen, maxLen;
bz_stream* strm = s->strm;
/* stuff that needs to be saved/restored */
Int32 i;
Int32 j;
Int32 t;
Int32 alphaSize;
Int32 nGroups;
Int32 nSelectors;
Int32 EOB;
Int32 groupNo;
Int32 groupPos;
Int32 nextSym;
Int32 nblockMAX;
Int32 nblock;
Int32 es;
Int32 N;
Int32 curr;
Int32 zt;
Int32 zn;
Int32 zvec;
Int32 zj;
Int32 gSel;
Int32 gMinlen;
Int32* gLimit;
Int32* gBase;
Int32* gPerm;
if (s->state == BZ_X_MAGIC_1) {
/*initialise the save area*/
s->save_i = 0;
s->save_j = 0;
s->save_t = 0;
s->save_alphaSize = 0;
s->save_nGroups = 0;
s->save_nSelectors = 0;
s->save_EOB = 0;
s->save_groupNo = 0;
s->save_groupPos = 0;
s->save_nextSym = 0;
s->save_nblockMAX = 0;
s->save_nblock = 0;
s->save_es = 0;
s->save_N = 0;
s->save_curr = 0;
s->save_zt = 0;
s->save_zn = 0;
s->save_zvec = 0;
s->save_zj = 0;
s->save_gSel = 0;
s->save_gMinlen = 0;
s->save_gLimit = NULL;
s->save_gBase = NULL;
s->save_gPerm = NULL;
}
/*restore from the save area*/
i = s->save_i;
j = s->save_j;
t = s->save_t;
alphaSize = s->save_alphaSize;
nGroups = s->save_nGroups;
nSelectors = s->save_nSelectors;
EOB = s->save_EOB;
groupNo = s->save_groupNo;
groupPos = s->save_groupPos;
nextSym = s->save_nextSym;
nblockMAX = s->save_nblockMAX;
nblock = s->save_nblock;
es = s->save_es;
N = s->save_N;
curr = s->save_curr;
zt = s->save_zt;
zn = s->save_zn;
zvec = s->save_zvec;
zj = s->save_zj;
gSel = s->save_gSel;
gMinlen = s->save_gMinlen;
gLimit = s->save_gLimit;
gBase = s->save_gBase;
gPerm = s->save_gPerm;
retVal = BZ_OK;
switch (s->state) {
GET_UCHAR(BZ_X_MAGIC_1, uc);
if (uc != BZ_HDR_B) RETURN(BZ_DATA_ERROR_MAGIC);
GET_UCHAR(BZ_X_MAGIC_2, uc);
if (uc != BZ_HDR_Z) RETURN(BZ_DATA_ERROR_MAGIC);
GET_UCHAR(BZ_X_MAGIC_3, uc)
if (uc != BZ_HDR_h) RETURN(BZ_DATA_ERROR_MAGIC);
GET_BITS(BZ_X_MAGIC_4, s->blockSize100k, 8)
if (s->blockSize100k < (BZ_HDR_0 + 1) ||
s->blockSize100k > (BZ_HDR_0 + 9)) RETURN(BZ_DATA_ERROR_MAGIC);
s->blockSize100k -= BZ_HDR_0;
if (s->smallDecompress) {
s->ll16 = BZALLOC( s->blockSize100k * 100000 * sizeof(UInt16) );
s->ll4 = BZALLOC(
((1 + s->blockSize100k * 100000) >> 1) * sizeof(UChar)
);
if (s->ll16 == NULL || s->ll4 == NULL) RETURN(BZ_MEM_ERROR);
} else {
s->tt = BZALLOC( s->blockSize100k * 100000 * sizeof(Int32) );
if (s->tt == NULL) RETURN(BZ_MEM_ERROR);
}
GET_UCHAR(BZ_X_BLKHDR_1, uc);
if (uc == 0x17) goto endhdr_2;
if (uc != 0x31) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_2, uc);
if (uc != 0x41) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_3, uc);
if (uc != 0x59) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_4, uc);
if (uc != 0x26) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_5, uc);
if (uc != 0x53) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_6, uc);
if (uc != 0x59) RETURN(BZ_DATA_ERROR);
s->currBlockNo++;
if (s->verbosity >= 2)
VPrintf1 ( "\n [%d: huff+mtf ", s->currBlockNo );
s->storedBlockCRC = 0;
GET_UCHAR(BZ_X_BCRC_1, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_2, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_3, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_4, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_BITS(BZ_X_RANDBIT, s->blockRandomised, 1);
s->origPtr = 0;
GET_UCHAR(BZ_X_ORIGPTR_1, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
GET_UCHAR(BZ_X_ORIGPTR_2, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
GET_UCHAR(BZ_X_ORIGPTR_3, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
if (s->origPtr < 0)
RETURN(BZ_DATA_ERROR);
if (s->origPtr > 10 + 100000*s->blockSize100k)
RETURN(BZ_DATA_ERROR);
/*--- Receive the mapping table ---*/
for (i = 0; i < 16; i++) {
GET_BIT(BZ_X_MAPPING_1, uc);
if (uc == 1)
s->inUse16[i] = True; else
s->inUse16[i] = False;
}
for (i = 0; i < 256; i++) s->inUse[i] = False;
for (i = 0; i < 16; i++)
if (s->inUse16[i])
for (j = 0; j < 16; j++) {
GET_BIT(BZ_X_MAPPING_2, uc);
if (uc == 1) s->inUse[i * 16 + j] = True;
}
makeMaps_d ( s );
if (s->nInUse == 0) RETURN(BZ_DATA_ERROR);
alphaSize = s->nInUse+2;
/*--- Now the selectors ---*/
GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
if (nGroups < 2 || nGroups > 6) RETURN(BZ_DATA_ERROR);
GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
for (i = 0; i < nSelectors; i++) {
j = 0;
while (True) {
GET_BIT(BZ_X_SELECTOR_3, uc);
if (uc == 0) break;
j++;
if (j >= nGroups) RETURN(BZ_DATA_ERROR);
}
s->selectorMtf[i] = j;
}
/*--- Undo the MTF values for the selectors. ---*/
{
UChar pos[BZ_N_GROUPS], tmp, v;
for (v = 0; v < nGroups; v++) pos[v] = v;
for (i = 0; i < nSelectors; i++) {
v = s->selectorMtf[i];
tmp = pos[v];
while (v > 0) { pos[v] = pos[v-1]; v--; }
pos[0] = tmp;
s->selector[i] = tmp;
}
}
/*--- Now the coding tables ---*/
for (t = 0; t < nGroups; t++) {
GET_BITS(BZ_X_CODING_1, curr, 5);
for (i = 0; i < alphaSize; i++) {
while (True) {
if (curr < 1 || curr > 20) RETURN(BZ_DATA_ERROR);
GET_BIT(BZ_X_CODING_2, uc);
if (uc == 0) break;
GET_BIT(BZ_X_CODING_3, uc);
if (uc == 0) curr++; else curr--;
}
s->len[t][i] = curr;
}
}
/*--- Create the Huffman decoding tables ---*/
for (t = 0; t < nGroups; t++) {
minLen = 32;
maxLen = 0;
for (i = 0; i < alphaSize; i++) {
if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
if (s->len[t][i] < minLen) minLen = s->len[t][i];
}
BZ2_hbCreateDecodeTables (
&(s->limit[t][0]),
&(s->base[t][0]),
&(s->perm[t][0]),
&(s->len[t][0]),
minLen, maxLen, alphaSize
);
s->minLens[t] = minLen;
}
/*--- Now the MTF values ---*/
EOB = s->nInUse+1;
nblockMAX = 100000 * s->blockSize100k;
groupNo = -1;
groupPos = 0;
for (i = 0; i <= 255; i++) s->unzftab[i] = 0;
/*-- MTF init --*/
{
Int32 ii, jj, kk;
kk = MTFA_SIZE-1;
for (ii = 256 / MTFL_SIZE - 1; ii >= 0; ii--) {
for (jj = MTFL_SIZE-1; jj >= 0; jj--) {
s->mtfa[kk] = (UChar)(ii * MTFL_SIZE + jj);
kk--;
}
s->mtfbase[ii] = kk + 1;
}
}
/*-- end MTF init --*/
nblock = 0;
GET_MTF_VAL(BZ_X_MTF_1, BZ_X_MTF_2, nextSym);
while (True) {
if (nextSym == EOB) break;
if (nextSym == BZ_RUNA || nextSym == BZ_RUNB) {
es = -1;
N = 1;
do {
if (nextSym == BZ_RUNA) es = es + (0+1) * N; else
if (nextSym == BZ_RUNB) es = es + (1+1) * N;
N = N * 2;
GET_MTF_VAL(BZ_X_MTF_3, BZ_X_MTF_4, nextSym);
}
while (nextSym == BZ_RUNA || nextSym == BZ_RUNB);
es++;
uc = s->seqToUnseq[ s->mtfa[s->mtfbase[0]] ];
s->unzftab[uc] += es;
if (s->smallDecompress)
while (es > 0) {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
s->ll16[nblock] = (UInt16)uc;
nblock++;
es--;
}
else
while (es > 0) {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
s->tt[nblock] = (UInt32)uc;
nblock++;
es--;
};
continue;
} else {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
/*-- uc = MTF ( nextSym-1 ) --*/
{
Int32 ii, jj, kk, pp, lno, off;
UInt32 nn;
nn = (UInt32)(nextSym - 1);
if (nn < MTFL_SIZE) {
/* avoid general-case expense */
pp = s->mtfbase[0];
uc = s->mtfa[pp+nn];
while (nn > 3) {
Int32 z = pp+nn;
s->mtfa[(z) ] = s->mtfa[(z)-1];
s->mtfa[(z)-1] = s->mtfa[(z)-2];
s->mtfa[(z)-2] = s->mtfa[(z)-3];
s->mtfa[(z)-3] = s->mtfa[(z)-4];
nn -= 4;
}
while (nn > 0) {
s->mtfa[(pp+nn)] = s->mtfa[(pp+nn)-1]; nn--;
};
s->mtfa[pp] = uc;
} else {
/* general case */
lno = nn / MTFL_SIZE;
off = nn % MTFL_SIZE;
pp = s->mtfbase[lno] + off;
uc = s->mtfa[pp];
while (pp > s->mtfbase[lno]) {
s->mtfa[pp] = s->mtfa[pp-1]; pp--;
};
s->mtfbase[lno]++;
while (lno > 0) {
s->mtfbase[lno]--;
s->mtfa[s->mtfbase[lno]]
= s->mtfa[s->mtfbase[lno-1] + MTFL_SIZE - 1];
lno--;
}
s->mtfbase[0]--;
s->mtfa[s->mtfbase[0]] = uc;
if (s->mtfbase[0] == 0) {
kk = MTFA_SIZE-1;
for (ii = 256 / MTFL_SIZE-1; ii >= 0; ii--) {
for (jj = MTFL_SIZE-1; jj >= 0; jj--) {
s->mtfa[kk] = s->mtfa[s->mtfbase[ii] + jj];
kk--;
}
s->mtfbase[ii] = kk + 1;
}
}
}
}
/*-- end uc = MTF ( nextSym-1 ) --*/
s->unzftab[s->seqToUnseq[uc]]++;
if (s->smallDecompress)
s->ll16[nblock] = (UInt16)(s->seqToUnseq[uc]); else
s->tt[nblock] = (UInt32)(s->seqToUnseq[uc]);
nblock++;
GET_MTF_VAL(BZ_X_MTF_5, BZ_X_MTF_6, nextSym);
continue;
}
}
/* Now we know what nblock is, we can do a better sanity
check on s->origPtr.
*/
if (s->origPtr < 0 || s->origPtr >= nblock)
RETURN(BZ_DATA_ERROR);
/*-- Set up cftab to facilitate generation of T^(-1) --*/
s->cftab[0] = 0;
for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1];
for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1];
for (i = 0; i <= 256; i++) {
if (s->cftab[i] < 0 || s->cftab[i] > nblock) {
/* s->cftab[i] can legitimately be == nblock */
RETURN(BZ_DATA_ERROR);
}
}
s->state_out_len = 0;
s->state_out_ch = 0;
BZ_INITIALISE_CRC ( s->calculatedBlockCRC );
s->state = BZ_X_OUTPUT;
if (s->verbosity >= 2) VPrintf0 ( "rt+rld" );
if (s->smallDecompress) {
/*-- Make a copy of cftab, used in generation of T --*/
for (i = 0; i <= 256; i++) s->cftabCopy[i] = s->cftab[i];
/*-- compute the T vector --*/
for (i = 0; i < nblock; i++) {
uc = (UChar)(s->ll16[i]);
SET_LL(i, s->cftabCopy[uc]);
s->cftabCopy[uc]++;
}
/*-- Compute T^(-1) by pointer reversal on T --*/
i = s->origPtr;
j = GET_LL(i);
do {
Int32 tmp = GET_LL(j);
SET_LL(j, i);
i = j;
j = tmp;
}
while (i != s->origPtr);
s->tPos = s->origPtr;
s->nblock_used = 0;
if (s->blockRandomised) {
BZ_RAND_INIT_MASK;
BZ_GET_SMALL(s->k0); s->nblock_used++;
BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK;
} else {
BZ_GET_SMALL(s->k0); s->nblock_used++;
}
} else {
/*-- compute the T^(-1) vector --*/
for (i = 0; i < nblock; i++) {
uc = (UChar)(s->tt[i] & 0xff);
s->tt[s->cftab[uc]] |= (i << 8);
s->cftab[uc]++;
}
s->tPos = s->tt[s->origPtr] >> 8;
s->nblock_used = 0;
if (s->blockRandomised) {
BZ_RAND_INIT_MASK;
BZ_GET_FAST(s->k0); s->nblock_used++;
BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK;
} else {
BZ_GET_FAST(s->k0); s->nblock_used++;
}
}
RETURN(BZ_OK);
endhdr_2:
GET_UCHAR(BZ_X_ENDHDR_2, uc);
if (uc != 0x72) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_3, uc);
if (uc != 0x45) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_4, uc);
if (uc != 0x38) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_5, uc);
if (uc != 0x50) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_6, uc);
if (uc != 0x90) RETURN(BZ_DATA_ERROR);
s->storedCombinedCRC = 0;
GET_UCHAR(BZ_X_CCRC_1, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_2, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_3, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_4, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
s->state = BZ_X_IDLE;
RETURN(BZ_STREAM_END);
default: AssertH ( False, 4001 );
}
AssertH ( False, 4002 );
save_state_and_return:
s->save_i = i;
s->save_j = j;
s->save_t = t;
s->save_alphaSize = alphaSize;
s->save_nGroups = nGroups;
s->save_nSelectors = nSelectors;
s->save_EOB = EOB;
s->save_groupNo = groupNo;
s->save_groupPos = groupPos;
s->save_nextSym = nextSym;
s->save_nblockMAX = nblockMAX;
s->save_nblock = nblock;
s->save_es = es;
s->save_N = N;
s->save_curr = curr;
s->save_zt = zt;
s->save_zn = zn;
s->save_zvec = zvec;
s->save_zj = zj;
s->save_gSel = gSel;
s->save_gMinlen = gMinlen;
s->save_gLimit = gLimit;
s->save_gBase = gBase;
s->save_gPerm = gPerm;
return retVal;
}
/*-------------------------------------------------------------*/
/*--- end decompress.c ---*/
/*-------------------------------------------------------------*/

176
contrib/vmap_extractor_v2/stormlib/bzip2/dlltest.c
View file

@ -1,176 +0,0 @@
/*
minibz2
libbz2.dll test program.
by Yoshioka Tsuneo(QWF00133@nifty.ne.jp/tsuneo-y@is.aist-nara.ac.jp)
This file is Public Domain.
welcome any email to me.
usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]
*/
#define BZ_IMPORT
#include <stdio.h>
#include <stdlib.h>
#include "bzlib.h"
#ifdef _WIN32
#include <io.h>
#endif
#ifdef _WIN32
#define BZ2_LIBNAME "libbz2-1.0.2.DLL"
#include <windows.h>
static int BZ2DLLLoaded = 0;
static HINSTANCE BZ2DLLhLib;
int BZ2DLLLoadLibrary(void)
{
HINSTANCE hLib;
if(BZ2DLLLoaded==1){return 0;}
hLib=LoadLibrary(BZ2_LIBNAME);
if(hLib == NULL){
fprintf(stderr,"Can't load %s\n",BZ2_LIBNAME);
return -1;
}
BZ2_bzlibVersion=GetProcAddress(hLib,"BZ2_bzlibVersion");
BZ2_bzopen=GetProcAddress(hLib,"BZ2_bzopen");
BZ2_bzdopen=GetProcAddress(hLib,"BZ2_bzdopen");
BZ2_bzread=GetProcAddress(hLib,"BZ2_bzread");
BZ2_bzwrite=GetProcAddress(hLib,"BZ2_bzwrite");
BZ2_bzflush=GetProcAddress(hLib,"BZ2_bzflush");
BZ2_bzclose=GetProcAddress(hLib,"BZ2_bzclose");
BZ2_bzerror=GetProcAddress(hLib,"BZ2_bzerror");
if (!BZ2_bzlibVersion || !BZ2_bzopen || !BZ2_bzdopen
|| !BZ2_bzread || !BZ2_bzwrite || !BZ2_bzflush
|| !BZ2_bzclose || !BZ2_bzerror) {
fprintf(stderr,"GetProcAddress failed.\n");
return -1;
}
BZ2DLLLoaded=1;
BZ2DLLhLib=hLib;
return 0;
}
int BZ2DLLFreeLibrary(void)
{
if(BZ2DLLLoaded==0){return 0;}
FreeLibrary(BZ2DLLhLib);
BZ2DLLLoaded=0;
}
#endif /* WIN32 */
void usage(void)
{
puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]");
}
int main(int argc,char *argv[])
{
int decompress = 0;
int level = 9;
char *fn_r = NULL;
char *fn_w = NULL;
#ifdef _WIN32
if(BZ2DLLLoadLibrary()<0){
fprintf(stderr,"Loading of %s failed. Giving up.\n", BZ2_LIBNAME);
exit(1);
}
printf("Loading of %s succeeded. Library version is %s.\n",
BZ2_LIBNAME, BZ2_bzlibVersion() );
#endif
while(++argv,--argc){
if(**argv =='-' || **argv=='/'){
char *p;
for(p=*argv+1;*p;p++){
if(*p=='d'){
decompress = 1;
}else if('1'<=*p && *p<='9'){
level = *p - '0';
}else{
usage();
exit(1);
}
}
}else{
break;
}
}
if(argc>=1){
fn_r = *argv;
argc--;argv++;
}else{
fn_r = NULL;
}
if(argc>=1){
fn_w = *argv;
argc--;argv++;
}else{
fn_w = NULL;
}
{
int len;
char buff[0x1000];
char mode[10];
if(decompress){
BZFILE *BZ2fp_r = NULL;
FILE *fp_w = NULL;
if(fn_w){
if((fp_w = fopen(fn_w,"wb"))==NULL){
printf("can't open [%s]\n",fn_w);
perror("reason:");
exit(1);
}
}else{
fp_w = stdout;
}
if((fn_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL)
|| (fn_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){
printf("can't bz2openstream\n");
exit(1);
}
while((len=BZ2_bzread(BZ2fp_r,buff,0x1000))>0){
fwrite(buff,1,len,fp_w);
}
BZ2_bzclose(BZ2fp_r);
if(fp_w != stdout) fclose(fp_w);
}else{
BZFILE *BZ2fp_w = NULL;
FILE *fp_r = NULL;
if(fn_r){
if((fp_r = fopen(fn_r,"rb"))==NULL){
printf("can't open [%s]\n",fn_r);
perror("reason:");
exit(1);
}
}else{
fp_r = stdin;
}
mode[0]='w';
mode[1] = '0' + level;
mode[2] = '\0';
if((fn_w == NULL && (BZ2fp_w = BZ2_bzdopen(fileno(stdout),mode))==NULL)
|| (fn_w !=NULL && (BZ2fp_w = BZ2_bzopen(fn_w,mode))==NULL)){
printf("can't bz2openstream\n");
exit(1);
}
while((len=fread(buff,1,0x1000,fp_r))>0){
BZ2_bzwrite(BZ2fp_w,buff,len);
}
BZ2_bzclose(BZ2fp_w);
if(fp_r!=stdin)fclose(fp_r);
}
}
#ifdef _WIN32
BZ2DLLFreeLibrary();
#endif
return 0;
}

93
contrib/vmap_extractor_v2/stormlib/bzip2/dlltest.dsp
View file

@ -1,93 +0,0 @@
# Microsoft Developer Studio Project File - Name="dlltest" - Package Owner=<4>
# Microsoft Developer Studio Generated Build File, Format Version 5.00
# ** 編集しないでください **
# TARGTYPE "Win32 (x86) Console Application" 0x0103
CFG=dlltest - Win32 Debug
!MESSAGE これは有効なﾒｲｸﾌｧｲﾙではありません。 このﾌﾟﾛｼﾞｪｸﾄをﾋﾞﾙﾄﾞするためには NMAKE を使用してください。
!MESSAGE [ﾒｲｸﾌｧｲﾙのｴｸｽﾎﾟｰﾄ] ｺﾏﾝﾄﾞを使用して実行してください
!MESSAGE
!MESSAGE NMAKE /f "dlltest.mak".
!MESSAGE
!MESSAGE NMAKE の実行時に構成を指定できます
!MESSAGE ｺﾏﾝﾄﾞ ﾗｲﾝ上でﾏｸﾛの設定を定義します。例:
!MESSAGE
!MESSAGE NMAKE /f "dlltest.mak" CFG="dlltest - Win32 Debug"
!MESSAGE
!MESSAGE 選択可能なﾋﾞﾙﾄﾞ ﾓｰﾄﾞ:
!MESSAGE
!MESSAGE "dlltest - Win32 Release" ("Win32 (x86) Console Application" 用)
!MESSAGE "dlltest - Win32 Debug" ("Win32 (x86) Console Application" 用)
!MESSAGE
# Begin Project
# PROP Scc_ProjName ""
# PROP Scc_LocalPath ""
CPP=cl.exe
RSC=rc.exe
!IF "$(CFG)" == "dlltest - Win32 Release"
# PROP BASE Use_MFC 0
# PROP BASE Use_Debug_Libraries 0
# PROP BASE Output_Dir "Release"
# PROP BASE Intermediate_Dir "Release"
# PROP BASE Target_Dir ""
# PROP Use_MFC 0
# PROP Use_Debug_Libraries 0
# PROP Output_Dir "Release"
# PROP Intermediate_Dir "Release"
# PROP Ignore_Export_Lib 0
# PROP Target_Dir ""
# ADD BASE CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
# ADD CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
# ADD BASE RSC /l 0x411 /d "NDEBUG"
# ADD RSC /l 0x411 /d "NDEBUG"
BSC32=bscmake.exe
# ADD BASE BSC32 /nologo
# ADD BSC32 /nologo
LINK32=link.exe
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386 /out:"minibz2.exe"
!ELSEIF "$(CFG)" == "dlltest - Win32 Debug"
# PROP BASE Use_MFC 0
# PROP BASE Use_Debug_Libraries 1
# PROP BASE Output_Dir "dlltest_"
# PROP BASE Intermediate_Dir "dlltest_"
# PROP BASE Target_Dir ""
# PROP Use_MFC 0
# PROP Use_Debug_Libraries 1
# PROP Output_Dir "dlltest_"
# PROP Intermediate_Dir "dlltest_"
# PROP Ignore_Export_Lib 0
# PROP Target_Dir ""
# ADD BASE CPP /nologo /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
# ADD CPP /nologo /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
# ADD BASE RSC /l 0x411 /d "_DEBUG"
# ADD RSC /l 0x411 /d "_DEBUG"
BSC32=bscmake.exe
# ADD BASE BSC32 /nologo
# ADD BSC32 /nologo
LINK32=link.exe
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /out:"minibz2.exe" /pdbtype:sept
!ENDIF
# Begin Target
# Name "dlltest - Win32 Release"
# Name "dlltest - Win32 Debug"
# Begin Source File
SOURCE=.\bzlib.h
# End Source File
# Begin Source File
SOURCE=.\dlltest.c
# End Source File
# End Target
# End Project

9
contrib/vmap_extractor_v2/stormlib/bzip2/entities.xml
View file

@ -1,9 +0,0 @@
<!-- misc. strings -->
<!ENTITY bz-url "http://www.bzip.org">
<!ENTITY bz-email "jseward@bzip.org">
<!ENTITY bz-lifespan "1996-2005">
<!ENTITY bz-version "1.0.3">
<!ENTITY bz-date "15 February 2005">
<!ENTITY manual-title "bzip2 Manual">

53
contrib/vmap_extractor_v2/stormlib/bzip2/format.pl
View file

@ -1,53 +0,0 @@
#!/usr/bin/perl -w
use strict;
# get command line values:
if ( $#ARGV !=1 ) {
die "Usage: $0 xml_infile xml_outfile\n";
}
my $infile = shift;
# check infile exists
die "Can't find file \"$infile\""
unless -f $infile;
# check we can read infile
if (! -r $infile) {
die "Can't read input $infile\n";
}
# check we can open infile
open( INFILE,"<$infile" ) or
die "Can't input $infile $!";
#my $outfile = 'fmt-manual.xml';
my $outfile = shift;
#print "Infile: $infile, Outfile: $outfile\n";
# check we can write to outfile
open( OUTFILE,">$outfile" ) or
die "Can't output $outfile $! for writing";
my ($prev, $curr, $str);
$prev = ''; $curr = '';
while ( <INFILE> ) {
print OUTFILE $prev;
$prev = $curr;
$curr = $_;
$str = '';
if ( $prev =~ /<programlisting>$|<screen>$/ ) {
chomp $prev;
$curr = join( '', $prev, "<![CDATA[", $curr );
$prev = '';
next;
}
elsif ( $curr =~ /<\/programlisting>|<\/screen>/ ) {
chomp $prev;
$curr = join( '', $prev, "]]>", $curr );
$prev = '';
next;
}
}
print OUTFILE $curr;
close INFILE;
close OUTFILE;
exit;

245
contrib/vmap_extractor_v2/stormlib/bzip2/huffman.c
View file

@ -1,245 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Huffman coding low-level stuff ---*/
/*--- huffman.c ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
#include "bzlib_private.h"
/*---------------------------------------------------*/
#define WEIGHTOF(zz0) ((zz0) & 0xffffff00)
#define DEPTHOF(zz1) ((zz1) & 0x000000ff)
#define MYMAX(zz2,zz3) ((zz2) > (zz3) ? (zz2) : (zz3))
#define ADDWEIGHTS(zw1,zw2) \
(WEIGHTOF(zw1)+WEIGHTOF(zw2)) | \
(1 + MYMAX(DEPTHOF(zw1),DEPTHOF(zw2)))
#define UPHEAP(z) \
{ \
Int32 zz, tmp; \
zz = z; tmp = heap[zz]; \
while (weight[tmp] < weight[heap[zz >> 1]]) { \
heap[zz] = heap[zz >> 1]; \
zz >>= 1; \
} \
heap[zz] = tmp; \
}
#define DOWNHEAP(z) \
{ \
Int32 zz, yy, tmp; \
zz = z; tmp = heap[zz]; \
while (True) { \
yy = zz << 1; \
if (yy > nHeap) break; \
if (yy < nHeap && \
weight[heap[yy+1]] < weight[heap[yy]]) \
yy++; \
if (weight[tmp] < weight[heap[yy]]) break; \
heap[zz] = heap[yy]; \
zz = yy; \
} \
heap[zz] = tmp; \
}
/*---------------------------------------------------*/
void BZ2_hbMakeCodeLengths ( UChar *len,
Int32 *freq,
Int32 alphaSize,
Int32 maxLen )
{
/*--
Nodes and heap entries run from 1. Entry 0
for both the heap and nodes is a sentinel.
--*/
Int32 nNodes, nHeap, n1, n2, i, j, k;
Bool tooLong;
Int32 heap [ BZ_MAX_ALPHA_SIZE + 2 ];
Int32 weight [ BZ_MAX_ALPHA_SIZE * 2 ];
Int32 parent [ BZ_MAX_ALPHA_SIZE * 2 ];
for (i = 0; i < alphaSize; i++)
weight[i+1] = (freq[i] == 0 ? 1 : freq[i]) << 8;
while (True) {
nNodes = alphaSize;
nHeap = 0;
heap[0] = 0;
weight[0] = 0;
parent[0] = -2;
for (i = 1; i <= alphaSize; i++) {
parent[i] = -1;
nHeap++;
heap[nHeap] = i;
UPHEAP(nHeap);
}
AssertH( nHeap < (BZ_MAX_ALPHA_SIZE+2), 2001 );
while (nHeap > 1) {
n1 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1);
n2 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1);
nNodes++;
parent[n1] = parent[n2] = nNodes;
weight[nNodes] = ADDWEIGHTS(weight[n1], weight[n2]);
parent[nNodes] = -1;
nHeap++;
heap[nHeap] = nNodes;
UPHEAP(nHeap);
}
AssertH( nNodes < (BZ_MAX_ALPHA_SIZE * 2), 2002 );
tooLong = False;
for (i = 1; i <= alphaSize; i++) {
j = 0;
k = i;
while (parent[k] >= 0) { k = parent[k]; j++; }
len[i-1] = j;
if (j > maxLen) tooLong = True;
}
if (! tooLong) break;
/* 17 Oct 04: keep-going condition for the following loop used
to be 'i < alphaSize', which missed the last element,
theoretically leading to the possibility of the compressor
looping. However, this count-scaling step is only needed if
one of the generated Huffman code words is longer than
maxLen, which up to and including version 1.0.2 was 20 bits,
which is extremely unlikely. In version 1.0.3 maxLen was
changed to 17 bits, which has minimal effect on compression
ratio, but does mean this scaling step is used from time to
time, enough to verify that it works.
This means that bzip2-1.0.3 and later will only produce
Huffman codes with a maximum length of 17 bits. However, in
order to preserve backwards compatibility with bitstreams
produced by versions pre-1.0.3, the decompressor must still
handle lengths of up to 20. */
for (i = 1; i <= alphaSize; i++) {
j = weight[i] >> 8;
j = 1 + (j / 2);
weight[i] = j << 8;
}
}
}
/*---------------------------------------------------*/
void BZ2_hbAssignCodes ( Int32 *code,
UChar *length,
Int32 minLen,
Int32 maxLen,
Int32 alphaSize )
{
Int32 n, vec, i;
vec = 0;
for (n = minLen; n <= maxLen; n++) {
for (i = 0; i < alphaSize; i++)
if (length[i] == n) { code[i] = vec; vec++; };
vec <<= 1;
}
}
/*---------------------------------------------------*/
void BZ2_hbCreateDecodeTables ( Int32 *limit,
Int32 *base,
Int32 *perm,
UChar *length,
Int32 minLen,
Int32 maxLen,
Int32 alphaSize )
{
Int32 pp, i, j, vec;
pp = 0;
for (i = minLen; i <= maxLen; i++)
for (j = 0; j < alphaSize; j++)
if (length[j] == i) { perm[pp] = j; pp++; };
for (i = 0; i < BZ_MAX_CODE_LEN; i++) base[i] = 0;
for (i = 0; i < alphaSize; i++) base[length[i]+1]++;
for (i = 1; i < BZ_MAX_CODE_LEN; i++) base[i] += base[i-1];
for (i = 0; i < BZ_MAX_CODE_LEN; i++) limit[i] = 0;
vec = 0;
for (i = minLen; i <= maxLen; i++) {
vec += (base[i+1] - base[i]);
limit[i] = vec-1;
vec <<= 1;
}
for (i = minLen + 1; i <= maxLen; i++)
base[i] = ((limit[i-1] + 1) << 1) - base[i];
}
/*-------------------------------------------------------------*/
/*--- end huffman.c ---*/
/*-------------------------------------------------------------*/

27
contrib/vmap_extractor_v2/stormlib/bzip2/libbz2.def
View file

@ -1,27 +0,0 @@
LIBRARY LIBBZ2
DESCRIPTION "libbzip2: library for data compression"
EXPORTS
BZ2_bzCompressInit
BZ2_bzCompress
BZ2_bzCompressEnd
BZ2_bzDecompressInit
BZ2_bzDecompress
BZ2_bzDecompressEnd
BZ2_bzReadOpen
BZ2_bzReadClose
BZ2_bzReadGetUnused
BZ2_bzRead
BZ2_bzWriteOpen
BZ2_bzWrite
BZ2_bzWriteClose
BZ2_bzWriteClose64
BZ2_bzBuffToBuffCompress
BZ2_bzBuffToBuffDecompress
BZ2_bzlibVersion
BZ2_bzopen
BZ2_bzdopen
BZ2_bzread
BZ2_bzwrite
BZ2_bzflush
BZ2_bzclose
BZ2_bzerror

130
contrib/vmap_extractor_v2/stormlib/bzip2/libbz2.dsp
View file

@ -1,130 +0,0 @@
# Microsoft Developer Studio Project File - Name="libbz2" - Package Owner=<4>
# Microsoft Developer Studio Generated Build File, Format Version 5.00
# ** 編集しないでください **
# TARGTYPE "Win32 (x86) Dynamic-Link Library" 0x0102
CFG=libbz2 - Win32 Debug
!MESSAGE これは有効なﾒｲｸﾌｧｲﾙではありません。 このﾌﾟﾛｼﾞｪｸﾄをﾋﾞﾙﾄﾞするためには NMAKE を使用してください。
!MESSAGE [ﾒｲｸﾌｧｲﾙのｴｸｽﾎﾟｰﾄ] ｺﾏﾝﾄﾞを使用して実行してください
!MESSAGE
!MESSAGE NMAKE /f "libbz2.mak".
!MESSAGE
!MESSAGE NMAKE の実行時に構成を指定できます
!MESSAGE ｺﾏﾝﾄﾞ ﾗｲﾝ上でﾏｸﾛの設定を定義します。例:
!MESSAGE
!MESSAGE NMAKE /f "libbz2.mak" CFG="libbz2 - Win32 Debug"
!MESSAGE
!MESSAGE 選択可能なﾋﾞﾙﾄﾞ ﾓｰﾄﾞ:
!MESSAGE
!MESSAGE "libbz2 - Win32 Release" ("Win32 (x86) Dynamic-Link Library" 用)
!MESSAGE "libbz2 - Win32 Debug" ("Win32 (x86) Dynamic-Link Library" 用)
!MESSAGE
# Begin Project
# PROP Scc_ProjName ""
# PROP Scc_LocalPath ""
CPP=cl.exe
MTL=midl.exe
RSC=rc.exe
!IF "$(CFG)" == "libbz2 - Win32 Release"
# PROP BASE Use_MFC 0
# PROP BASE Use_Debug_Libraries 0
# PROP BASE Output_Dir "Release"
# PROP BASE Intermediate_Dir "Release"
# PROP BASE Target_Dir ""
# PROP Use_MFC 0
# PROP Use_Debug_Libraries 0
# PROP Output_Dir "Release"
# PROP Intermediate_Dir "Release"
# PROP Ignore_Export_Lib 0
# PROP Target_Dir ""
# ADD BASE CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /YX /FD /c
# ADD CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /YX /FD /c
# ADD BASE MTL /nologo /D "NDEBUG" /mktyplib203 /o NUL /win32
# ADD MTL /nologo /D "NDEBUG" /mktyplib203 /o NUL /win32
# ADD BASE RSC /l 0x411 /d "NDEBUG"
# ADD RSC /l 0x411 /d "NDEBUG"
BSC32=bscmake.exe
# ADD BASE BSC32 /nologo
# ADD BSC32 /nologo
LINK32=link.exe
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /machine:I386
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /machine:I386 /out:"libbz2.dll"
!ELSEIF "$(CFG)" == "libbz2 - Win32 Debug"
# PROP BASE Use_MFC 0
# PROP BASE Use_Debug_Libraries 1
# PROP BASE Output_Dir "Debug"
# PROP BASE Intermediate_Dir "Debug"
# PROP BASE Target_Dir ""
# PROP Use_MFC 0
# PROP Use_Debug_Libraries 1
# PROP Output_Dir "Debug"
# PROP Intermediate_Dir "Debug"
# PROP Ignore_Export_Lib 0
# PROP Target_Dir ""
# ADD BASE CPP /nologo /MTd /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /YX /FD /c
# ADD CPP /nologo /MTd /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /YX /FD /c
# ADD BASE MTL /nologo /D "_DEBUG" /mktyplib203 /o NUL /win32
# ADD MTL /nologo /D "_DEBUG" /mktyplib203 /o NUL /win32
# ADD BASE RSC /l 0x411 /d "_DEBUG"
# ADD RSC /l 0x411 /d "_DEBUG"
BSC32=bscmake.exe
# ADD BASE BSC32 /nologo
# ADD BSC32 /nologo
LINK32=link.exe
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /debug /machine:I386 /pdbtype:sept
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /debug /machine:I386 /out:"libbz2.dll" /pdbtype:sept
!ENDIF
# Begin Target
# Name "libbz2 - Win32 Release"
# Name "libbz2 - Win32 Debug"
# Begin Source File
SOURCE=.\blocksort.c
# End Source File
# Begin Source File
SOURCE=.\bzlib.c
# End Source File
# Begin Source File
SOURCE=.\bzlib.h
# End Source File
# Begin Source File
SOURCE=.\bzlib_private.h
# End Source File
# Begin Source File
SOURCE=.\compress.c
# End Source File
# Begin Source File
SOURCE=.\crctable.c
# End Source File
# Begin Source File
SOURCE=.\decompress.c
# End Source File
# Begin Source File
SOURCE=.\huffman.c
# End Source File
# Begin Source File
SOURCE=.\libbz2.def
# End Source File
# Begin Source File
SOURCE=.\randtable.c
# End Source File
# End Target
# End Project

63
contrib/vmap_extractor_v2/stormlib/bzip2/makefile.msc
View file

@ -1,63 +0,0 @@
# Makefile for Microsoft Visual C++ 6.0
# usage: nmake -f makefile.msc
# K.M. Syring (syring@gsf.de)
# Fixed up by JRS for bzip2-0.9.5d release.
CC=cl
CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 -nologo
OBJS= blocksort.obj \
huffman.obj \
crctable.obj \
randtable.obj \
compress.obj \
decompress.obj \
bzlib.obj
all: lib bzip2 test
bzip2: lib
$(CC) $(CFLAGS) -o bzip2 bzip2.c libbz2.lib setargv.obj
$(CC) $(CFLAGS) -o bzip2recover bzip2recover.c
lib: $(OBJS)
lib /out:libbz2.lib $(OBJS)
test: bzip2
type words1
.\\bzip2 -1 < sample1.ref > sample1.rb2
.\\bzip2 -2 < sample2.ref > sample2.rb2
.\\bzip2 -3 < sample3.ref > sample3.rb2
.\\bzip2 -d < sample1.bz2 > sample1.tst
.\\bzip2 -d < sample2.bz2 > sample2.tst
.\\bzip2 -ds < sample3.bz2 > sample3.tst
@echo All six of the fc's should find no differences.
@echo If fc finds an error on sample3.bz2, this could be
@echo because WinZip's 'TAR file smart CR/LF conversion'
@echo is too clever for its own good. Disable this option.
@echo The correct size for sample3.ref is 120,244. If it
@echo is 150,251, WinZip has messed it up.
fc sample1.bz2 sample1.rb2
fc sample2.bz2 sample2.rb2
fc sample3.bz2 sample3.rb2
fc sample1.tst sample1.ref
fc sample2.tst sample2.ref
fc sample3.tst sample3.ref
clean:
del *.obj
del libbz2.lib
del bzip2.exe
del bzip2recover.exe
del sample1.rb2
del sample2.rb2
del sample3.rb2
del sample1.tst
del sample2.tst
del sample3.tst
.c.obj:
$(CC) $(CFLAGS) -c $*.c -o $*.obj

2687
contrib/vmap_extractor_v2/stormlib/bzip2/manual.html
View file

File diff suppressed because it is too large Load diff

BIN
contrib/vmap_extractor_v2/stormlib/bzip2/manual.pdf
View file

Binary file not shown.

68244
contrib/vmap_extractor_v2/stormlib/bzip2/manual.ps
View file

File diff suppressed because it is too large Load diff

2966
contrib/vmap_extractor_v2/stormlib/bzip2/manual.xml
View file

File diff suppressed because it is too large Load diff

16
contrib/vmap_extractor_v2/stormlib/bzip2/mk251.c
View file

@ -1,16 +0,0 @@
/* Spew out a long sequence of the byte 251. When fed to bzip2
versions 1.0.0 or 1.0.1, causes it to die with internal error
1007 in blocksort.c. This assertion misses an extremely rare
case, which is fixed in this version (1.0.2) and above.
*/
#include <stdio.h>
int main ()
{
int i;
for (i = 0; i < 48500000 ; i++)
putchar(251);
return 0;
}

124
contrib/vmap_extractor_v2/stormlib/bzip2/randtable.c
View file

@ -1,124 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Table for randomising repetitive blocks ---*/
/*--- randtable.c ---*/
/*-------------------------------------------------------------*/
/*--
This file is a part of bzip2 and/or libbzip2, a program and
library for lossless, block-sorting data compression.
Copyright (C) 1996-2005 Julian R Seward. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, Cambridge, UK.
jseward@bzip.org
bzip2/libbzip2 version 1.0 of 21 March 2000
This program is based on (at least) the work of:
Mike Burrows
David Wheeler
Peter Fenwick
Alistair Moffat
Radford Neal
Ian H. Witten
Robert Sedgewick
Jon L. Bentley
For more information on these sources, see the manual.
--*/
#include "bzlib_private.h"
/*---------------------------------------------*/
Int32 BZ2_rNums[512] = {
619, 720, 127, 481, 931, 816, 813, 233, 566, 247,
985, 724, 205, 454, 863, 491, 741, 242, 949, 214,
733, 859, 335, 708, 621, 574, 73, 654, 730, 472,
419, 436, 278, 496, 867, 210, 399, 680, 480, 51,
878, 465, 811, 169, 869, 675, 611, 697, 867, 561,
862, 687, 507, 283, 482, 129, 807, 591, 733, 623,
150, 238, 59, 379, 684, 877, 625, 169, 643, 105,
170, 607, 520, 932, 727, 476, 693, 425, 174, 647,
73, 122, 335, 530, 442, 853, 695, 249, 445, 515,
909, 545, 703, 919, 874, 474, 882, 500, 594, 612,
641, 801, 220, 162, 819, 984, 589, 513, 495, 799,
161, 604, 958, 533, 221, 400, 386, 867, 600, 782,
382, 596, 414, 171, 516, 375, 682, 485, 911, 276,
98, 553, 163, 354, 666, 933, 424, 341, 533, 870,
227, 730, 475, 186, 263, 647, 537, 686, 600, 224,
469, 68, 770, 919, 190, 373, 294, 822, 808, 206,
184, 943, 795, 384, 383, 461, 404, 758, 839, 887,
715, 67, 618, 276, 204, 918, 873, 777, 604, 560,
951, 160, 578, 722, 79, 804, 96, 409, 713, 940,
652, 934, 970, 447, 318, 353, 859, 672, 112, 785,
645, 863, 803, 350, 139, 93, 354, 99, 820, 908,
609, 772, 154, 274, 580, 184, 79, 626, 630, 742,
653, 282, 762, 623, 680, 81, 927, 626, 789, 125,
411, 521, 938, 300, 821, 78, 343, 175, 128, 250,
170, 774, 972, 275, 999, 639, 495, 78, 352, 126,
857, 956, 358, 619, 580, 124, 737, 594, 701, 612,
669, 112, 134, 694, 363, 992, 809, 743, 168, 974,
944, 375, 748, 52, 600, 747, 642, 182, 862, 81,
344, 805, 988, 739, 511, 655, 814, 334, 249, 515,
897, 955, 664, 981, 649, 113, 974, 459, 893, 228,
433, 837, 553, 268, 926, 240, 102, 654, 459, 51,
686, 754, 806, 760, 493, 403, 415, 394, 687, 700,
946, 670, 656, 610, 738, 392, 760, 799, 887, 653,
978, 321, 576, 617, 626, 502, 894, 679, 243, 440,
680, 879, 194, 572, 640, 724, 926, 56, 204, 700,
707, 151, 457, 449, 797, 195, 791, 558, 945, 679,
297, 59, 87, 824, 713, 663, 412, 693, 342, 606,
134, 108, 571, 364, 631, 212, 174, 643, 304, 329,
343, 97, 430, 751, 497, 314, 983, 374, 822, 928,
140, 206, 73, 263, 980, 736, 876, 478, 430, 305,
170, 514, 364, 692, 829, 82, 855, 953, 676, 246,
369, 970, 294, 750, 807, 827, 150, 790, 288, 923,
804, 378, 215, 828, 592, 281, 565, 555, 710, 82,
896, 831, 547, 261, 524, 462, 293, 465, 502, 56,
661, 821, 976, 991, 658, 869, 905, 758, 745, 193,
768, 550, 608, 933, 378, 286, 215, 979, 792, 961,
61, 688, 793, 644, 986, 403, 106, 366, 905, 644,
372, 567, 466, 434, 645, 210, 389, 550, 919, 135,
780, 773, 635, 389, 707, 100, 626, 958, 165, 504,
920, 176, 193, 713, 857, 265, 203, 50, 668, 108,
645, 990, 626, 197, 510, 357, 358, 850, 858, 364,
936, 638
};
/*-------------------------------------------------------------*/
/*--- end randtable.c ---*/
/*-------------------------------------------------------------*/

BIN
contrib/vmap_extractor_v2/stormlib/bzip2/sample1.bz2
View file

Binary file not shown.

BIN
contrib/vmap_extractor_v2/stormlib/bzip2/sample1.ref
View file

Binary file not shown.

BIN
contrib/vmap_extractor_v2/stormlib/bzip2/sample2.bz2
View file

Binary file not shown.

BIN
contrib/vmap_extractor_v2/stormlib/bzip2/sample2.ref
View file

Binary file not shown.

BIN
contrib/vmap_extractor_v2/stormlib/bzip2/sample3.bz2
View file

Binary file not shown.

30007
contrib/vmap_extractor_v2/stormlib/bzip2/sample3.ref
View file

File diff suppressed because it is too large Load diff

39
contrib/vmap_extractor_v2/stormlib/bzip2/spewG.c
View file

@ -1,39 +0,0 @@
/* spew out a thoroughly gigantic file designed so that bzip2
can compress it reasonably rapidly. This is to help test
support for large files (> 2GB) in a reasonable amount of time.
I suggest you use the undocumented --exponential option to
bzip2 when compressing the resulting file; this saves a bit of
time. Note: *don't* bother with --exponential when compressing
Real Files; it'll just waste a lot of CPU time :-)
(but is otherwise harmless).
*/
#define _FILE_OFFSET_BITS 64
#include <stdio.h>
#include <stdlib.h>
/* The number of megabytes of junk to spew out (roughly) */
#define MEGABYTES 5000
#define N_BUF 1000000
char buf[N_BUF];
int main ( int argc, char** argv )
{
int ii, kk, p;
srandom(1);
setbuffer ( stdout, buf, N_BUF );
for (kk = 0; kk < MEGABYTES * 515; kk+=3) {
p = 25+random()%50;
for (ii = 0; ii < p; ii++)
printf ( "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" );
for (ii = 0; ii < p-1; ii++)
printf ( "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" );
for (ii = 0; ii < p+1; ii++)
printf ( "ccccccccccccccccccccccccccccccccccccc" );
}
fflush(stdout);
return 0;
}

126
contrib/vmap_extractor_v2/stormlib/bzip2/unzcrash.c
View file

@ -1,126 +0,0 @@
/* A test program written to test robustness to decompression of
corrupted data. Usage is
unzcrash filename
and the program will read the specified file, compress it (in memory),
and then repeatedly decompress it, each time with a different bit of
the compressed data inverted, so as to test all possible one-bit errors.
This should not cause any invalid memory accesses. If it does,
I want to know about it!
p.s. As you can see from the above description, the process is
incredibly slow. A file of size eg 5KB will cause it to run for
many hours.
*/
#include <stdio.h>
#include <assert.h>
#include "bzlib.h"
#define M_BLOCK 1000000
typedef unsigned char uchar;
#define M_BLOCK_OUT (M_BLOCK + 1000000)
uchar inbuf[M_BLOCK];
uchar outbuf[M_BLOCK_OUT];
uchar zbuf[M_BLOCK + 600 + (M_BLOCK / 100)];
int nIn, nOut, nZ;
static char *bzerrorstrings[] = {
"OK"
,"SEQUENCE_ERROR"
,"PARAM_ERROR"
,"MEM_ERROR"
,"DATA_ERROR"
,"DATA_ERROR_MAGIC"
,"IO_ERROR"
,"UNEXPECTED_EOF"
,"OUTBUFF_FULL"
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
};
void flip_bit ( int bit )
{
int byteno = bit / 8;
int bitno = bit % 8;
uchar mask = 1 << bitno;
//fprintf ( stderr, "(byte %d bit %d mask %d)",
// byteno, bitno, (int)mask );
zbuf[byteno] ^= mask;
}
int main ( int argc, char** argv )
{
FILE* f;
int r;
int bit;
int i;
if (argc != 2) {
fprintf ( stderr, "usage: unzcrash filename\n" );
return 1;
}
f = fopen ( argv[1], "r" );
if (!f) {
fprintf ( stderr, "unzcrash: can't open %s\n", argv[1] );
return 1;
}
nIn = fread ( inbuf, 1, M_BLOCK, f );
fprintf ( stderr, "%d bytes read\n", nIn );
nZ = M_BLOCK;
r = BZ2_bzBuffToBuffCompress (
zbuf, &nZ, inbuf, nIn, 9, 0, 30 );
assert (r == BZ_OK);
fprintf ( stderr, "%d after compression\n", nZ );
for (bit = 0; bit < nZ*8; bit++) {
fprintf ( stderr, "bit %d ", bit );
flip_bit ( bit );
nOut = M_BLOCK_OUT;
r = BZ2_bzBuffToBuffDecompress (
outbuf, &nOut, zbuf, nZ, 0, 0 );
fprintf ( stderr, " %d %s ", r, bzerrorstrings[-r] );
if (r != BZ_OK) {
fprintf ( stderr, "\n" );
} else {
if (nOut != nIn) {
fprintf(stderr, "nIn/nOut mismatch %d %d\n", nIn, nOut );
return 1;
} else {
for (i = 0; i < nOut; i++)
if (inbuf[i] != outbuf[i]) {
fprintf(stderr, "mismatch at %d\n", i );
return 1;
}
if (i == nOut) fprintf(stderr, "really ok!\n" );
}
}
flip_bit ( bit );
}
#if 0
assert (nOut == nIn);
for (i = 0; i < nOut; i++) {
if (inbuf[i] != outbuf[i]) {
fprintf ( stderr, "difference at %d !\n", i );
return 1;
}
}
#endif
fprintf ( stderr, "all ok\n" );
return 0;
}

5
contrib/vmap_extractor_v2/stormlib/bzip2/words0
View file

@ -1,5 +0,0 @@
If compilation produces errors, or a large number of warnings,
please read README.COMPILATION.PROBLEMS -- you might be able to
adjust the flags in this Makefile to improve matters.

4
contrib/vmap_extractor_v2/stormlib/bzip2/words1
View file

@ -1,4 +0,0 @@
Doing 6 tests (3 compress, 3 uncompress) ...
If there's a problem, things might stop at this point.

5
contrib/vmap_extractor_v2/stormlib/bzip2/words2
View file

@ -1,5 +0,0 @@
Checking test results. If any of the four "cmp"s which follow
report any differences, something is wrong. If you can't easily
figure out what, please let me know (jseward@acm.org).

23
contrib/vmap_extractor_v2/stormlib/bzip2/words3
View file

@ -1,23 +0,0 @@
If you got this far and the "cmp"s didn't complain, it looks
like you're in business.
To install in /usr/bin, /usr/lib, /usr/man and /usr/include, type
make install
To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type
make install PREFIX=/xxx/yyy
If you are (justifiably) paranoid and want to see what 'make install'
is going to do, you can first do
make -n install or
make -n install PREFIX=/xxx/yyy respectively.
The -n instructs make to show the commands it would execute, but
not actually execute them.
Instructions for use are in the preformatted manual page, in the file
bzip2.txt. For more detailed documentation, read the full manual.
It is available in Postscript form (manual.ps), PDF form (manual.pdf),
and HTML form (manual_toc.html).
You can also do "bzip2 --help" to see some helpful information.
"bzip2 -L" displays the software license.

99
contrib/vmap_extractor_v2/stormlib/bzip2/xmlproc.sh
View file

@ -1,99 +0,0 @@
#!/bin/bash
# see the README in this directory for usage etc.
usage() {
echo '';
echo 'Usage: xmlproc.sh -[option] <filename.xml>';
echo 'Specify a target from:';
echo '-v verify xml file conforms to dtd';
echo '-html output in html format (single file)';
echo '-ps output in postscript format';
echo '-pdf output in pdf format';
exit;
}
if test $# -ne 2; then
usage
fi
# assign the variable for the output type
action=$1; shift
# assign the output filename
xmlfile=$1; shift
# and check user input it correct
if !(test -f $xmlfile); then
echo "No such file: $xmlfile";
exit;
fi
# some other stuff we will use
OUT=output
xsl_fo=bz-fo.xsl
xsl_html=bz-html.xsl
basename=$xmlfile
basename=${basename//'.xml'/''}
fofile="${basename}.fo"
htmlfile="${basename}.html"
pdffile="${basename}.pdf"
psfile="${basename}.ps"
xmlfmtfile="${basename}.fmt"
# first process the xmlfile with CDATA tags
./format.pl $xmlfile $xmlfmtfile
# so the shell knows where the catalogs live
export XML_CATALOG_FILES=/etc/xml/catalog
# post-processing tidy up
cleanup() {
echo "Cleaning up: # $@"
while [ $# != 0 ]
do
arg=$1; shift;
echo " deleting $arg";
rm $arg
done
}
case $action in
-v)
flags='--noout --xinclude --noblanks --postvalid'
dtd='--dtdvalid http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd'
xmllint $flags $dtd $xmlfmtfile 2> $OUT
egrep 'error' $OUT
rm $OUT
;;
-html)
echo "Creating $htmlfile ..."
xsltproc --nonet --xinclude -o $htmlfile $xsl_html $xmlfmtfile
cleanup $xmlfmtfile
;;
-pdf)
echo "Creating $pdffile ..."
xsltproc --nonet --xinclude -o $fofile $xsl_fo $xmlfmtfile
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
cleanup $OUT $xmlfmtfile *.aux *.fo *.log *.out
;;
-ps)
echo "Creating $psfile ..."
xsltproc --nonet --xinclude -o $fofile $xsl_fo $xmlfmtfile
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
pdftops $pdffile $psfile
cleanup $OUT $xmlfmtfile $pdffile *.aux *.fo *.log *.out
# passivetex is broken, so we can't go this route yet.
# xmltex $fofile >$OUT </dev/null
# xmltex $fofile >$OUT </dev/null
# xmltex $fofile >$OUT </dev/null
# dvips -R -q -o bzip-manual.ps *.dvi
;;
*)
usage
;;
esac

1453
contrib/vmap_extractor_v2/stormlib/huffman/huff.cpp
View file

File diff suppressed because it is too large Load diff

142
contrib/vmap_extractor_v2/stormlib/huffman/huff.h
View file

@ -1,142 +0,0 @@
/*****************************************************************************/
/* huffman.h Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Description : */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* xx.xx.xx 1.00 Lad The first version of huffman.h */
/* 03.05.03 2.00 Lad Added compression */
/* 08.12.03 2.01 Dan High-memory handling (> 0x80000000) */
/*****************************************************************************/
#ifndef __HUFFMAN_H__
#define __HUFFMAN_H__
#include "../StormPort.h"
//-----------------------------------------------------------------------------
// Defines
#define INSERT_ITEM 1
#define SWITCH_ITEMS 2 // Switch the item1 and item2
#define PTR_NOT(ptr) (THTreeItem *)(~(DWORD_PTR)(ptr))
#define PTR_PTR(ptr) ((THTreeItem *)(ptr))
#define PTR_INT(ptr) (LONG_PTR)(ptr)
#ifndef NULL
#define NULL 0
#endif
//-----------------------------------------------------------------------------
// Structures and classes
// Input stream for Huffmann decompression
class TInputStream
{
public:
unsigned long GetBit();
unsigned long Get7Bits();
unsigned long Get8Bits();
unsigned char * pbInBuffer; // 00 - Input data
unsigned long dwBitBuff; // 04 - Input bit buffer
unsigned int nBits; // 08 - Number of bits remaining in 'dwValue'
};
// Output stream for Huffmann compression
class TOutputStream
{
public:
void PutBits(unsigned long dwBuff, unsigned int nPutBits);
unsigned char * pbOutBuffer; // 00 : Output buffer
unsigned long dwOutSize; // 04 : Size of output buffer
unsigned char * pbOutPos; // 08 : Current output position
unsigned long dwBitBuff; // 0C : Bit buffer
unsigned long nBits; // 10 : Number of bits in the bit buffer
};
// Huffmann tree item (?)
struct THTreeItem
{
public:
THTreeItem * Call1501DB70(THTreeItem * pLast);
THTreeItem * GetPrevItem(LONG_PTR value);
void ClearItemLinks();
void RemoveItem();
THTreeItem * next; // 00 - Pointer to next THTreeItem
THTreeItem * prev; // 04 - Pointer to prev THTreeItem (< 0 if none)
unsigned long dcmpByte; // 08 - Index of this item in item pointer array, decompressed byte value
unsigned long byteValue; // 0C - Some byte value
THTreeItem * parent; // 10 - Pointer to parent THTreeItem (NULL if none)
THTreeItem * child; // 14 - Pointer to child THTreeItem
int addressMultiplier; // -1 if object on negative address (>0x80000000), +1 if positive
};
// Structure used for quick decompress. The 'bitCount' contains number of bits
// and byte value contains result decompressed byte value.
// After each walk through Huffman tree are filled all entries which are
// multiplies of number of bits loaded from input stream. These entries
// contain number of bits and result value. At the next 7 bits is tested this
// structure first. If corresponding entry found, decompression routine will
// not walk through Huffman tree and directly stores output byte to output stream.
struct TQDecompress
{
unsigned long offs00; // 00 - 1 if resolved
unsigned long nBits; // 04 - Bit count
union
{
unsigned long dcmpByte; // 08 - Byte value for decompress (if bitCount <= 7)
THTreeItem * pItem; // 08 - THTreeItem (if number of bits is greater than 7
};
};
// Structure for Huffman tree (Size 0x3674 bytes). Because I'm not expert
// for the decompression, I do not know actually if the class is really a Hufmann
// tree. If someone knows the decompression details, please let me know
class THuffmannTree
{
public:
THuffmannTree();
void InitTree(bool bCompression);
void BuildTree(unsigned int nCmpType);
// void ModifyTree(unsigned long dwIndex);
// void UninitTree();
// void Call15007010(Bit32 dwInLength, THTreeItem * item);
THTreeItem * Call1500E740(unsigned int nValue);
void Call1500E820(THTreeItem * pItem);
unsigned int DoCompression(TOutputStream * os, unsigned char * pbInBuffer, int nInLength, int nCmpType);
unsigned int DoDecompression(unsigned char * pbOutBuffer, unsigned int dwOutLength, TInputStream * is);
unsigned long bIsCmp0; // 0000 - 1 if compression type 0
unsigned long offs0004; // 0004 - Some flag
THTreeItem items0008[0x203]; // 0008 - HTree items
//- Sometimes used as HTree item -----------
THTreeItem * pItem3050; // 3050 - Always NULL (?)
THTreeItem * pItem3054; // 3054 - Pointer to Huffman tree item
THTreeItem * pItem3058; // 3058 - Pointer to Huffman tree item (< 0 if invalid)
//- Sometimes used as HTree item -----------
THTreeItem * pItem305C; // 305C - Usually NULL
THTreeItem * pFirst; // 3060 - Pointer to top (first) Huffman tree item
THTreeItem * pLast; // 3064 - Pointer to bottom (last) Huffman tree item (< 0 if invalid)
unsigned long nItems; // 3068 - Number of used HTree items
//-------------------------------------------
THTreeItem * items306C[0x102]; // 306C - THTreeItem pointer array
TQDecompress qd3474[0x80]; // 3474 - Array for quick decompression
int addressMultiplier; // -1 if object on negative address (>0x80000000), +1 if positive
static unsigned char Table1502A630[];// Some table
};
#endif // __HUFFMAN_H__

72
contrib/vmap_extractor_v2/stormlib/pklib/crc32.c
View file

@ -1,72 +0,0 @@
/*****************************************************************************/
/* crc32.c Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Pkware Data Compression Library Version 1.11 */
/* Dissassembled method crc32 - cdecl version */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 09.04.03 1.00 Lad The first version of crc32.c */
/* 02.05.03 1.00 Lad Stress test done */
/*****************************************************************************/
#include "pklib.h"
static char CopyRight[] = "PKWARE Data Compression Library for Win32\r\n"
"Copyright 1989-1995 PKWARE Inc. All Rights Reserved\r\n"
"Patent No. 5,051,745\r\n"
"PKWARE Data Compression Library Reg. U.S. Pat. and Tm. Off.\r\n"
"Version 1.11\r\n";
static unsigned long crc_table[] =
{
0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419, 0x706AF48F, 0xE963A535, 0x9E6495A3,
0x0EDB8832, 0x79DCB8A4, 0xE0D5E91E, 0x97D2D988, 0x09B64C2B, 0x7EB17CBD, 0xE7B82D07, 0x90BF1D91,
0x1DB71064, 0x6AB020F2, 0xF3B97148, 0x84BE41DE, 0x1ADAD47D, 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7,
0x136C9856, 0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, 0x14015C4F, 0x63066CD9, 0xFA0F3D63, 0x8D080DF5,
0x3B6E20C8, 0x4C69105E, 0xD56041E4, 0xA2677172, 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B,
0x35B5A8FA, 0x42B2986C, 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3, 0x45DF5C75, 0xDCD60DCF, 0xABD13D59,
0x26D930AC, 0x51DE003A, 0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, 0xCFBA9599, 0xB8BDA50F,
0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924, 0x2F6F7C87, 0x58684C11, 0xC1611DAB, 0xB6662D3D,
0x76DC4190, 0x01DB7106, 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F, 0x9FBFE4A5, 0xE8B8D433,
0x7807C9A2, 0x0F00F934, 0x9609A88E, 0xE10E9818, 0x7F6A0DBB, 0x086D3D2D, 0x91646C97, 0xE6635C01,
0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, 0x6C0695ED, 0x1B01A57B, 0x8208F4C1, 0xF50FC457,
0x65B0D9C6, 0x12B7E950, 0x8BBEB8EA, 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3, 0xFBD44C65,
0x4DB26158, 0x3AB551CE, 0xA3BC0074, 0xD4BB30E2, 0x4ADFA541, 0x3DD895D7, 0xA4D1C46D, 0xD3D6F4FB,
0x4369E96A, 0x346ED9FC, 0xAD678846, 0xDA60B8D0, 0x44042D73, 0x33031DE5, 0xAA0A4C5F, 0xDD0D7CC9,
0x5005713C, 0x270241AA, 0xBE0B1010, 0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, 0xCE61E49F,
0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17, 0x2EB40D81, 0xB7BD5C3B, 0xC0BA6CAD,
0xEDB88320, 0x9ABFB3B6, 0x03B6E20C, 0x74B1D29A, 0xEAD54739, 0x9DD277AF, 0x04DB2615, 0x73DC1683,
0xE3630B12, 0x94643B84, 0x0D6D6A3E, 0x7A6A5AA8, 0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1,
0xF00F9344, 0x8708A3D2, 0x1E01F268, 0x6906C2FE, 0xF762575D, 0x806567CB, 0x196C3671, 0x6E6B06E7,
0xFED41B76, 0x89D32BE0, 0x10DA7A5A, 0x67DD4ACC, 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5,
0xD6D6A3E8, 0xA1D1937E, 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1, 0xA6BC5767, 0x3FB506DD, 0x48B2364B,
0xD80D2BDA, 0xAF0A1B4C, 0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, 0x316E8EEF, 0x4669BE79,
0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236, 0xCC0C7795, 0xBB0B4703, 0x220216B9, 0x5505262F,
0xC5BA3BBE, 0xB2BD0B28, 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, 0xB5D0CF31, 0x2CD99E8B, 0x5BDEAE1D,
0x9B64C2B0, 0xEC63F226, 0x756AA39C, 0x026D930A, 0x9C0906A9, 0xEB0E363F, 0x72076785, 0x05005713,
0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, 0x0CB61B38, 0x92D28E9B, 0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21,
0x86D3D2D4, 0xF1D4E242, 0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1, 0x18B74777,
0x88085AE6, 0xFF0F6A70, 0x66063BCA, 0x11010B5C, 0x8F659EFF, 0xF862AE69, 0x616BFFD3, 0x166CCF45,
0xA00AE278, 0xD70DD2EE, 0x4E048354, 0x3903B3C2, 0xA7672661, 0xD06016F7, 0x4969474D, 0x3E6E77DB,
0xAED16A4A, 0xD9D65ADC, 0x40DF0B66, 0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, 0x47B2CF7F, 0x30B5FFE9,
0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605, 0xCDD70693, 0x54DE5729, 0x23D967BF,
0xB3667A2E, 0xC4614AB8, 0x5D681B02, 0x2A6F2B94, 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B, 0x2D02EF8D
};
unsigned long PKEXPORT crc32pk(char * buffer, unsigned int * psize, unsigned long * old_crc)
{
unsigned int size = *psize;
unsigned long ch;
unsigned long crc_value = *old_crc;
while(size-- != 0)
{
ch = *buffer++ ^ (char)crc_value;
crc_value >>= 8;
crc_value = crc_table[ch & 0x0FF] ^ crc_value;
}
return crc_value;
}

72
contrib/vmap_extractor_v2/stormlib/pklib/crc32_pk.c
View file

@ -1,72 +0,0 @@
/*****************************************************************************/
/* crc32.c Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Pkware Data Compression Library Version 1.11 */
/* Dissassembled method crc32 - cdecl version */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 09.04.03 1.00 Lad The first version of crc32.c */
/* 02.05.03 1.00 Lad Stress test done */
/*****************************************************************************/
#include "pklib.h"
static char CopyRight[] = "PKWARE Data Compression Library for Win32\r\n"
"Copyright 1989-1995 PKWARE Inc. All Rights Reserved\r\n"
"Patent No. 5,051,745\r\n"
"PKWARE Data Compression Library Reg. U.S. Pat. and Tm. Off.\r\n"
"Version 1.11\r\n";
static unsigned long crc_table[] =
{
0x00000000, 0x77073096, 0xEE0E612C, 0x990951BA, 0x076DC419, 0x706AF48F, 0xE963A535, 0x9E6495A3,
0x0EDB8832, 0x79DCB8A4, 0xE0D5E91E, 0x97D2D988, 0x09B64C2B, 0x7EB17CBD, 0xE7B82D07, 0x90BF1D91,
0x1DB71064, 0x6AB020F2, 0xF3B97148, 0x84BE41DE, 0x1ADAD47D, 0x6DDDE4EB, 0xF4D4B551, 0x83D385C7,
0x136C9856, 0x646BA8C0, 0xFD62F97A, 0x8A65C9EC, 0x14015C4F, 0x63066CD9, 0xFA0F3D63, 0x8D080DF5,
0x3B6E20C8, 0x4C69105E, 0xD56041E4, 0xA2677172, 0x3C03E4D1, 0x4B04D447, 0xD20D85FD, 0xA50AB56B,
0x35B5A8FA, 0x42B2986C, 0xDBBBC9D6, 0xACBCF940, 0x32D86CE3, 0x45DF5C75, 0xDCD60DCF, 0xABD13D59,
0x26D930AC, 0x51DE003A, 0xC8D75180, 0xBFD06116, 0x21B4F4B5, 0x56B3C423, 0xCFBA9599, 0xB8BDA50F,
0x2802B89E, 0x5F058808, 0xC60CD9B2, 0xB10BE924, 0x2F6F7C87, 0x58684C11, 0xC1611DAB, 0xB6662D3D,
0x76DC4190, 0x01DB7106, 0x98D220BC, 0xEFD5102A, 0x71B18589, 0x06B6B51F, 0x9FBFE4A5, 0xE8B8D433,
0x7807C9A2, 0x0F00F934, 0x9609A88E, 0xE10E9818, 0x7F6A0DBB, 0x086D3D2D, 0x91646C97, 0xE6635C01,
0x6B6B51F4, 0x1C6C6162, 0x856530D8, 0xF262004E, 0x6C0695ED, 0x1B01A57B, 0x8208F4C1, 0xF50FC457,
0x65B0D9C6, 0x12B7E950, 0x8BBEB8EA, 0xFCB9887C, 0x62DD1DDF, 0x15DA2D49, 0x8CD37CF3, 0xFBD44C65,
0x4DB26158, 0x3AB551CE, 0xA3BC0074, 0xD4BB30E2, 0x4ADFA541, 0x3DD895D7, 0xA4D1C46D, 0xD3D6F4FB,
0x4369E96A, 0x346ED9FC, 0xAD678846, 0xDA60B8D0, 0x44042D73, 0x33031DE5, 0xAA0A4C5F, 0xDD0D7CC9,
0x5005713C, 0x270241AA, 0xBE0B1010, 0xC90C2086, 0x5768B525, 0x206F85B3, 0xB966D409, 0xCE61E49F,
0x5EDEF90E, 0x29D9C998, 0xB0D09822, 0xC7D7A8B4, 0x59B33D17, 0x2EB40D81, 0xB7BD5C3B, 0xC0BA6CAD,
0xEDB88320, 0x9ABFB3B6, 0x03B6E20C, 0x74B1D29A, 0xEAD54739, 0x9DD277AF, 0x04DB2615, 0x73DC1683,
0xE3630B12, 0x94643B84, 0x0D6D6A3E, 0x7A6A5AA8, 0xE40ECF0B, 0x9309FF9D, 0x0A00AE27, 0x7D079EB1,
0xF00F9344, 0x8708A3D2, 0x1E01F268, 0x6906C2FE, 0xF762575D, 0x806567CB, 0x196C3671, 0x6E6B06E7,
0xFED41B76, 0x89D32BE0, 0x10DA7A5A, 0x67DD4ACC, 0xF9B9DF6F, 0x8EBEEFF9, 0x17B7BE43, 0x60B08ED5,
0xD6D6A3E8, 0xA1D1937E, 0x38D8C2C4, 0x4FDFF252, 0xD1BB67F1, 0xA6BC5767, 0x3FB506DD, 0x48B2364B,
0xD80D2BDA, 0xAF0A1B4C, 0x36034AF6, 0x41047A60, 0xDF60EFC3, 0xA867DF55, 0x316E8EEF, 0x4669BE79,
0xCB61B38C, 0xBC66831A, 0x256FD2A0, 0x5268E236, 0xCC0C7795, 0xBB0B4703, 0x220216B9, 0x5505262F,
0xC5BA3BBE, 0xB2BD0B28, 0x2BB45A92, 0x5CB36A04, 0xC2D7FFA7, 0xB5D0CF31, 0x2CD99E8B, 0x5BDEAE1D,
0x9B64C2B0, 0xEC63F226, 0x756AA39C, 0x026D930A, 0x9C0906A9, 0xEB0E363F, 0x72076785, 0x05005713,
0x95BF4A82, 0xE2B87A14, 0x7BB12BAE, 0x0CB61B38, 0x92D28E9B, 0xE5D5BE0D, 0x7CDCEFB7, 0x0BDBDF21,
0x86D3D2D4, 0xF1D4E242, 0x68DDB3F8, 0x1FDA836E, 0x81BE16CD, 0xF6B9265B, 0x6FB077E1, 0x18B74777,
0x88085AE6, 0xFF0F6A70, 0x66063BCA, 0x11010B5C, 0x8F659EFF, 0xF862AE69, 0x616BFFD3, 0x166CCF45,
0xA00AE278, 0xD70DD2EE, 0x4E048354, 0x3903B3C2, 0xA7672661, 0xD06016F7, 0x4969474D, 0x3E6E77DB,
0xAED16A4A, 0xD9D65ADC, 0x40DF0B66, 0x37D83BF0, 0xA9BCAE53, 0xDEBB9EC5, 0x47B2CF7F, 0x30B5FFE9,
0xBDBDF21C, 0xCABAC28A, 0x53B39330, 0x24B4A3A6, 0xBAD03605, 0xCDD70693, 0x54DE5729, 0x23D967BF,
0xB3667A2E, 0xC4614AB8, 0x5D681B02, 0x2A6F2B94, 0xB40BBE37, 0xC30C8EA1, 0x5A05DF1B, 0x2D02EF8D
};
unsigned long PKEXPORT crc32pk(char * buffer, unsigned int * psize, unsigned long * old_crc)
{
unsigned int size = *psize;
unsigned long ch;
unsigned long crc_value = *old_crc;
while(size-- != 0)
{
ch = *buffer++ ^ (char)crc_value;
crc_value >>= 8;
crc_value = crc_table[ch & 0x0FF] ^ crc_value;
}
return crc_value;
}

480
contrib/vmap_extractor_v2/stormlib/pklib/explode.c
View file

@ -1,480 +0,0 @@
/*****************************************************************************/
/* explode.c Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Implode function of PKWARE Data Compression library */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 11.03.03 1.00 Lad Splitted from Pkware.cpp */
/* 08.04.03 1.01 Lad Renamed to explode.c to be compatible with pklib */
/* 02.05.03 1.01 Lad Stress test done */
/*****************************************************************************/
#include <assert.h>
#include <string.h>
#include "pklib.h"
//-----------------------------------------------------------------------------
// Tables
static unsigned char DistBits[] =
{
0x02, 0x04, 0x04, 0x05, 0x05, 0x05, 0x05, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06,
0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07,
0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07,
0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08
};
static unsigned char DistCode[] =
{
0x03, 0x0D, 0x05, 0x19, 0x09, 0x11, 0x01, 0x3E, 0x1E, 0x2E, 0x0E, 0x36, 0x16, 0x26, 0x06, 0x3A,
0x1A, 0x2A, 0x0A, 0x32, 0x12, 0x22, 0x42, 0x02, 0x7C, 0x3C, 0x5C, 0x1C, 0x6C, 0x2C, 0x4C, 0x0C,
0x74, 0x34, 0x54, 0x14, 0x64, 0x24, 0x44, 0x04, 0x78, 0x38, 0x58, 0x18, 0x68, 0x28, 0x48, 0x08,
0xF0, 0x70, 0xB0, 0x30, 0xD0, 0x50, 0x90, 0x10, 0xE0, 0x60, 0xA0, 0x20, 0xC0, 0x40, 0x80, 0x00
};
static unsigned char ExLenBits[] =
{
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08
};
static unsigned short LenBase[] =
{
0x0000, 0x0001, 0x0002, 0x0003, 0x0004, 0x0005, 0x0006, 0x0007,
0x0008, 0x000A, 0x000E, 0x0016, 0x0026, 0x0046, 0x0086, 0x0106
};
static unsigned char LenBits[] =
{
0x03, 0x02, 0x03, 0x03, 0x04, 0x04, 0x04, 0x05, 0x05, 0x05, 0x05, 0x06, 0x06, 0x06, 0x07, 0x07
};
static unsigned char LenCode[] =
{
0x05, 0x03, 0x01, 0x06, 0x0A, 0x02, 0x0C, 0x14, 0x04, 0x18, 0x08, 0x30, 0x10, 0x20, 0x40, 0x00
};
static unsigned char ChBitsAsc[] =
{
0x0B, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x08, 0x07, 0x0C, 0x0C, 0x07, 0x0C, 0x0C,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0D, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x04, 0x0A, 0x08, 0x0C, 0x0A, 0x0C, 0x0A, 0x08, 0x07, 0x07, 0x08, 0x09, 0x07, 0x06, 0x07, 0x08,
0x07, 0x06, 0x07, 0x07, 0x07, 0x07, 0x08, 0x07, 0x07, 0x08, 0x08, 0x0C, 0x0B, 0x07, 0x09, 0x0B,
0x0C, 0x06, 0x07, 0x06, 0x06, 0x05, 0x07, 0x08, 0x08, 0x06, 0x0B, 0x09, 0x06, 0x07, 0x06, 0x06,
0x07, 0x0B, 0x06, 0x06, 0x06, 0x07, 0x09, 0x08, 0x09, 0x09, 0x0B, 0x08, 0x0B, 0x09, 0x0C, 0x08,
0x0C, 0x05, 0x06, 0x06, 0x06, 0x05, 0x06, 0x06, 0x06, 0x05, 0x0B, 0x07, 0x05, 0x06, 0x05, 0x05,
0x06, 0x0A, 0x05, 0x05, 0x05, 0x05, 0x08, 0x07, 0x08, 0x08, 0x0A, 0x0B, 0x0B, 0x0C, 0x0C, 0x0C,
0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D,
0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D,
0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x0D, 0x0C, 0x0D, 0x0D, 0x0D, 0x0C, 0x0D, 0x0D, 0x0D, 0x0C, 0x0D, 0x0D, 0x0D, 0x0D, 0x0C, 0x0D,
0x0D, 0x0D, 0x0C, 0x0C, 0x0C, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D
};
static unsigned short ChCodeAsc[] =
{
0x0490, 0x0FE0, 0x07E0, 0x0BE0, 0x03E0, 0x0DE0, 0x05E0, 0x09E0,
0x01E0, 0x00B8, 0x0062, 0x0EE0, 0x06E0, 0x0022, 0x0AE0, 0x02E0,
0x0CE0, 0x04E0, 0x08E0, 0x00E0, 0x0F60, 0x0760, 0x0B60, 0x0360,
0x0D60, 0x0560, 0x1240, 0x0960, 0x0160, 0x0E60, 0x0660, 0x0A60,
0x000F, 0x0250, 0x0038, 0x0260, 0x0050, 0x0C60, 0x0390, 0x00D8,
0x0042, 0x0002, 0x0058, 0x01B0, 0x007C, 0x0029, 0x003C, 0x0098,
0x005C, 0x0009, 0x001C, 0x006C, 0x002C, 0x004C, 0x0018, 0x000C,
0x0074, 0x00E8, 0x0068, 0x0460, 0x0090, 0x0034, 0x00B0, 0x0710,
0x0860, 0x0031, 0x0054, 0x0011, 0x0021, 0x0017, 0x0014, 0x00A8,
0x0028, 0x0001, 0x0310, 0x0130, 0x003E, 0x0064, 0x001E, 0x002E,
0x0024, 0x0510, 0x000E, 0x0036, 0x0016, 0x0044, 0x0030, 0x00C8,
0x01D0, 0x00D0, 0x0110, 0x0048, 0x0610, 0x0150, 0x0060, 0x0088,
0x0FA0, 0x0007, 0x0026, 0x0006, 0x003A, 0x001B, 0x001A, 0x002A,
0x000A, 0x000B, 0x0210, 0x0004, 0x0013, 0x0032, 0x0003, 0x001D,
0x0012, 0x0190, 0x000D, 0x0015, 0x0005, 0x0019, 0x0008, 0x0078,
0x00F0, 0x0070, 0x0290, 0x0410, 0x0010, 0x07A0, 0x0BA0, 0x03A0,
0x0240, 0x1C40, 0x0C40, 0x1440, 0x0440, 0x1840, 0x0840, 0x1040,
0x0040, 0x1F80, 0x0F80, 0x1780, 0x0780, 0x1B80, 0x0B80, 0x1380,
0x0380, 0x1D80, 0x0D80, 0x1580, 0x0580, 0x1980, 0x0980, 0x1180,
0x0180, 0x1E80, 0x0E80, 0x1680, 0x0680, 0x1A80, 0x0A80, 0x1280,
0x0280, 0x1C80, 0x0C80, 0x1480, 0x0480, 0x1880, 0x0880, 0x1080,
0x0080, 0x1F00, 0x0F00, 0x1700, 0x0700, 0x1B00, 0x0B00, 0x1300,
0x0DA0, 0x05A0, 0x09A0, 0x01A0, 0x0EA0, 0x06A0, 0x0AA0, 0x02A0,
0x0CA0, 0x04A0, 0x08A0, 0x00A0, 0x0F20, 0x0720, 0x0B20, 0x0320,
0x0D20, 0x0520, 0x0920, 0x0120, 0x0E20, 0x0620, 0x0A20, 0x0220,
0x0C20, 0x0420, 0x0820, 0x0020, 0x0FC0, 0x07C0, 0x0BC0, 0x03C0,
0x0DC0, 0x05C0, 0x09C0, 0x01C0, 0x0EC0, 0x06C0, 0x0AC0, 0x02C0,
0x0CC0, 0x04C0, 0x08C0, 0x00C0, 0x0F40, 0x0740, 0x0B40, 0x0340,
0x0300, 0x0D40, 0x1D00, 0x0D00, 0x1500, 0x0540, 0x0500, 0x1900,
0x0900, 0x0940, 0x1100, 0x0100, 0x1E00, 0x0E00, 0x0140, 0x1600,
0x0600, 0x1A00, 0x0E40, 0x0640, 0x0A40, 0x0A00, 0x1200, 0x0200,
0x1C00, 0x0C00, 0x1400, 0x0400, 0x1800, 0x0800, 0x1000, 0x0000
};
//-----------------------------------------------------------------------------
// Local variables
static char Copyright[] = "PKWARE Data Compression Library for Win32\r\n"
"Copyright 1989-1995 PKWARE Inc. All Rights Reserved\r\n"
"Patent No. 5,051,745\r\n"
"PKWARE Data Compression Library Reg. U.S. Pat. and Tm. Off.\r\n"
"Version 1.11\r\n";
//-----------------------------------------------------------------------------
// Local functions
// Copies a block to another location
static void lmemcpy(void * trg, const void * src, size_t count)
{
memcpy(trg, src, count);
}
static void GenDecodeTabs(long count, unsigned char * bits, unsigned char * pCode, unsigned char * buffer2)
{
long i;
for(i = count-1; i >= 0; i--) // EBX - count
{
unsigned long idx1 = pCode[i];
unsigned long idx2 = 1 << bits[i];
do
{
buffer2[idx1] = (unsigned char)i;
idx1 += idx2;
}
while(idx1 < 0x100);
}
}
static void GenAscTabs(TDcmpStruct * pWork)
{
unsigned short * pChCodeAsc = &ChCodeAsc[0xFF];
unsigned long acc, add;
unsigned short count;
for(count = 0x00FF; pChCodeAsc >= ChCodeAsc; pChCodeAsc--, count--)
{
unsigned char * pChBitsAsc = pWork->ChBitsAsc + count;
unsigned char bits_asc = *pChBitsAsc;
if(bits_asc <= 8)
{
add = (1 << bits_asc);
acc = *pChCodeAsc;
do
{
pWork->offs2C34[acc] = (unsigned char)count;
acc += add;
}
while(acc < 0x100);
}
else if((acc = (*pChCodeAsc & 0xFF)) != 0)
{
pWork->offs2C34[acc] = 0xFF;
if(*pChCodeAsc & 0x3F)
{
bits_asc -= 4;
*pChBitsAsc = bits_asc;
add = (1 << bits_asc);
acc = *pChCodeAsc >> 4;
do
{
pWork->offs2D34[acc] = (unsigned char)count;
acc += add;
}
while(acc < 0x100);
}
else
{
bits_asc -= 6;
*pChBitsAsc = bits_asc;
add = (1 << bits_asc);
acc = *pChCodeAsc >> 6;
do
{
pWork->offs2E34[acc] = (unsigned char)count;
acc += add;
}
while(acc < 0x80);
}
}
else
{
bits_asc -= 8;
*pChBitsAsc = bits_asc;
add = (1 << bits_asc);
acc = *pChCodeAsc >> 8;
do
{
pWork->offs2EB4[acc] = (unsigned char)count;
acc += add;
}
while(acc < 0x100);
}
}
}
//-----------------------------------------------------------------------------
// Skips given number of bits in bit buffer. Result is stored in pWork->bit_buff
// If no data in input buffer, returns true
static int WasteBits(TDcmpStruct * pWork, unsigned long nBits)
{
// If number of bits required is less than number of (bits in the buffer) ?
if(nBits <= pWork->extra_bits)
{
pWork->extra_bits -= nBits;
pWork->bit_buff >>= nBits;
return 0;
}
// Load input buffer if necessary
pWork->bit_buff >>= pWork->extra_bits;
if(pWork->in_pos == pWork->in_bytes)
{
pWork->in_pos = sizeof(pWork->in_buff);
if((pWork->in_bytes = pWork->read_buf((char *)pWork->in_buff, &pWork->in_pos, pWork->param)) == 0)
return 1;
pWork->in_pos = 0;
}
// Update bit buffer
pWork->bit_buff |= (pWork->in_buff[pWork->in_pos++] << 8);
pWork->bit_buff >>= (nBits - pWork->extra_bits);
pWork->extra_bits = (pWork->extra_bits - nBits) + 8;
return 0;
}
//-----------------------------------------------------------------------------
// Returns : 0x000 - 0x0FF : One byte from compressed file.
// 0x100 - 0x305 : Copy previous block (0x100 = 1 byte)
// 0x306 : Out of buffer (?)
static unsigned long DecodeLit(TDcmpStruct * pWork)
{
unsigned long nBits; // Number of bits to skip
unsigned long value; // Position in buffers
// Test the current bit in byte buffer. If is not set, simply return the next byte.
if(pWork->bit_buff & 1)
{
// Skip current bit in the buffer
if(WasteBits(pWork, 1))
return 0x306;
// The next bits are position in buffers
value = pWork->position2[(pWork->bit_buff & 0xFF)];
// Get number of bits to skip
if(WasteBits(pWork, pWork->LenBits[value]))
return 0x306;
if((nBits = pWork->ExLenBits[value]) != 0)
{
unsigned long val2 = pWork->bit_buff & ((1 << nBits) - 1);
if(WasteBits(pWork, nBits))
{
if((value + val2) != 0x10E)
return 0x306;
}
value = pWork->LenBase[value] + val2;
}
return value + 0x100; // Return number of bytes to repeat
}
// Waste one bit
if(WasteBits(pWork, 1))
return 0x306;
// If the binary compression type, read 8 bits and return them as one byte.
if(pWork->ctype == CMP_BINARY)
{
value = pWork->bit_buff & 0xFF;
if(WasteBits(pWork, 8))
return 0x306;
return value;
}
// When ASCII compression ...
if(pWork->bit_buff & 0xFF)
{
value = pWork->offs2C34[pWork->bit_buff & 0xFF];
if(value == 0xFF)
{
if(pWork->bit_buff & 0x3F)
{
if(WasteBits(pWork, 4))
return 0x306;
value = pWork->offs2D34[pWork->bit_buff & 0xFF];
}
else
{
if(WasteBits(pWork, 6))
return 0x306;
value = pWork->offs2E34[pWork->bit_buff & 0x7F];
}
}
}
else
{
if(WasteBits(pWork, 8))
return 0x306;
value = pWork->offs2EB4[pWork->bit_buff & 0xFF];
}
return WasteBits(pWork, pWork->ChBitsAsc[value]) ? 0x306 : value;
}
//-----------------------------------------------------------------------------
// Retrieves the number of bytes to move back
static unsigned long DecodeDist(TDcmpStruct * pWork, unsigned long dwLength)
{
unsigned long pos = pWork->position1[(pWork->bit_buff & 0xFF)];
unsigned long nSkip = pWork->DistBits[pos]; // Number of bits to skip
// Skip the appropriate number of bits
if(WasteBits(pWork, nSkip) == 1)
return 0;
if(dwLength == 2)
{
pos = (pos << 2) | (pWork->bit_buff & 0x03);
if(WasteBits(pWork, 2) == 1)
return 0;
}
else
{
pos = (pos << pWork->dsize_bits) | (pWork->bit_buff & pWork->dsize_mask);
// Skip the bits
if(WasteBits(pWork, pWork->dsize_bits) == 1)
return 0;
}
return pos+1;
}
static unsigned long Expand(TDcmpStruct * pWork)
{
unsigned int copyBytes; // Number of bytes to copy
unsigned long oneByte; // One byte from compressed file
unsigned long dwResult;
pWork->outputPos = 0x1000; // Initialize output buffer position
// If end of data or error, terminate decompress
while((dwResult = oneByte = DecodeLit(pWork)) < 0x305)
{
// If one byte is greater than 0x100, means "Repeat n - 0xFE bytes"
if(oneByte >= 0x100)
{
unsigned char * source; // ECX
unsigned char * target; // EDX
unsigned long copyLength = oneByte - 0xFE;
unsigned long moveBack;
// Get length of data to copy
if((moveBack = DecodeDist(pWork, copyLength)) == 0)
{
dwResult = 0x306;
break;
}
// Target and source pointer
target = &pWork->out_buff[pWork->outputPos];
source = target - moveBack;
pWork->outputPos += copyLength;
while(copyLength-- > 0)
*target++ = *source++;
}
else
pWork->out_buff[pWork->outputPos++] = (unsigned char)oneByte;
// If number of extracted bytes has reached 1/2 of output buffer,
// flush output buffer.
if(pWork->outputPos >= 0x2000)
{
// Copy decompressed data into user buffer
copyBytes = 0x1000;
pWork->write_buf((char *)&pWork->out_buff[0x1000], &copyBytes, pWork->param);
// If there are some data left, keep them alive
lmemcpy(pWork->out_buff, &pWork->out_buff[0x1000], pWork->outputPos - 0x1000);
pWork->outputPos -= 0x1000;
}
}
copyBytes = pWork->outputPos - 0x1000;
pWork->write_buf((char *)&pWork->out_buff[0x1000], &copyBytes, pWork->param);
return dwResult;
}
//-----------------------------------------------------------------------------
// Main exploding function.
unsigned int explode(
unsigned int (*read_buf)(char *buf, unsigned int *size, void *param),
void (*write_buf)(char *buf, unsigned int *size, void *param),
char *work_buf,
void *param)
{
TDcmpStruct * pWork = (TDcmpStruct *)work_buf;
// Set the whole work buffer to zeros
memset(pWork, 0, sizeof(TDcmpStruct));
// Initialize work struct and load compressed data
pWork->read_buf = read_buf;
pWork->write_buf = write_buf;
pWork->param = param;
pWork->in_pos = sizeof(pWork->in_buff);
pWork->in_bytes = pWork->read_buf((char *)pWork->in_buff, &pWork->in_pos, pWork->param);
if(pWork->in_bytes <= 4)
return CMP_BAD_DATA;
pWork->ctype = pWork->in_buff[0]; // Get the compression type
pWork->dsize_bits = pWork->in_buff[1]; // Get the dictionary size
pWork->bit_buff = pWork->in_buff[2]; // Initialize 16-bit bit buffer
pWork->extra_bits = 0; // Extra (over 8) bits
pWork->in_pos = 3; // Position in input buffer
// Test for the valid dictionary size
if(4 > pWork->dsize_bits || pWork->dsize_bits > 6)
return CMP_INVALID_DICTSIZE;
pWork->dsize_mask = 0xFFFF >> (0x10 - pWork->dsize_bits); // Shifted by 'sar' instruction
if(pWork->ctype != CMP_BINARY)
{
if(pWork->ctype != CMP_ASCII)
return CMP_INVALID_MODE;
lmemcpy(pWork->ChBitsAsc, ChBitsAsc, sizeof(pWork->ChBitsAsc));
GenAscTabs(pWork);
}
lmemcpy(pWork->LenBits, LenBits, sizeof(pWork->LenBits));
GenDecodeTabs(0x10, pWork->LenBits, LenCode, pWork->position2);
lmemcpy(pWork->ExLenBits, ExLenBits, sizeof(pWork->ExLenBits));
lmemcpy(pWork->LenBase, LenBase, sizeof(pWork->LenBase));
lmemcpy(pWork->DistBits, DistBits, sizeof(pWork->DistBits));
GenDecodeTabs(0x40, pWork->DistBits, DistCode, pWork->position1);
if(Expand(pWork) != 0x306)
return CMP_NO_ERROR;
return CMP_ABORT;
}

674
contrib/vmap_extractor_v2/stormlib/pklib/implode.c
View file

@ -1,674 +0,0 @@
/*****************************************************************************/
/* implode.c Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Implode function of PKWARE Data Compression library */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 11.04.03 1.00 Lad First version of implode.c */
/* 02.05.03 1.00 Lad Stress test done */
/*****************************************************************************/
#include <assert.h>
#include <string.h>
#include "pklib.h"
#if ((1200 < _MSC_VER) && (_MSC_VER < 1400))
#pragma optimize("", off) // Fucking Microsoft VS.NET 2003 compiler !!!
// (_MSC_VER=1310)
#endif
//-----------------------------------------------------------------------------
// Defines
#define DICT_OFFSET 0x204
#define UNCMP_OFFSET (pWork->dsize_bytes + DICT_OFFSET)
//-----------------------------------------------------------------------------
// Tables
static unsigned char DistBits[] =
{
0x02, 0x04, 0x04, 0x05, 0x05, 0x05, 0x05, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x06,
0x06, 0x06, 0x06, 0x06, 0x06, 0x06, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07,
0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07, 0x07,
0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08
};
static unsigned char DistCode[] =
{
0x03, 0x0D, 0x05, 0x19, 0x09, 0x11, 0x01, 0x3E, 0x1E, 0x2E, 0x0E, 0x36, 0x16, 0x26, 0x06, 0x3A,
0x1A, 0x2A, 0x0A, 0x32, 0x12, 0x22, 0x42, 0x02, 0x7C, 0x3C, 0x5C, 0x1C, 0x6C, 0x2C, 0x4C, 0x0C,
0x74, 0x34, 0x54, 0x14, 0x64, 0x24, 0x44, 0x04, 0x78, 0x38, 0x58, 0x18, 0x68, 0x28, 0x48, 0x08,
0xF0, 0x70, 0xB0, 0x30, 0xD0, 0x50, 0x90, 0x10, 0xE0, 0x60, 0xA0, 0x20, 0xC0, 0x40, 0x80, 0x00
};
static unsigned char ExLenBits[] =
{
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08
};
static unsigned char LenBits[] =
{
0x03, 0x02, 0x03, 0x03, 0x04, 0x04, 0x04, 0x05, 0x05, 0x05, 0x05, 0x06, 0x06, 0x06, 0x07, 0x07
};
static unsigned char LenCode[] =
{
0x05, 0x03, 0x01, 0x06, 0x0A, 0x02, 0x0C, 0x14, 0x04, 0x18, 0x08, 0x30, 0x10, 0x20, 0x40, 0x00
};
static unsigned char ChBitsAsc[] =
{
0x0B, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x08, 0x07, 0x0C, 0x0C, 0x07, 0x0C, 0x0C,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0D, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x04, 0x0A, 0x08, 0x0C, 0x0A, 0x0C, 0x0A, 0x08, 0x07, 0x07, 0x08, 0x09, 0x07, 0x06, 0x07, 0x08,
0x07, 0x06, 0x07, 0x07, 0x07, 0x07, 0x08, 0x07, 0x07, 0x08, 0x08, 0x0C, 0x0B, 0x07, 0x09, 0x0B,
0x0C, 0x06, 0x07, 0x06, 0x06, 0x05, 0x07, 0x08, 0x08, 0x06, 0x0B, 0x09, 0x06, 0x07, 0x06, 0x06,
0x07, 0x0B, 0x06, 0x06, 0x06, 0x07, 0x09, 0x08, 0x09, 0x09, 0x0B, 0x08, 0x0B, 0x09, 0x0C, 0x08,
0x0C, 0x05, 0x06, 0x06, 0x06, 0x05, 0x06, 0x06, 0x06, 0x05, 0x0B, 0x07, 0x05, 0x06, 0x05, 0x05,
0x06, 0x0A, 0x05, 0x05, 0x05, 0x05, 0x08, 0x07, 0x08, 0x08, 0x0A, 0x0B, 0x0B, 0x0C, 0x0C, 0x0C,
0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D,
0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D,
0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C, 0x0C,
0x0D, 0x0C, 0x0D, 0x0D, 0x0D, 0x0C, 0x0D, 0x0D, 0x0D, 0x0C, 0x0D, 0x0D, 0x0D, 0x0D, 0x0C, 0x0D,
0x0D, 0x0D, 0x0C, 0x0C, 0x0C, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D
};
static unsigned short ChCodeAsc[] =
{
0x0490, 0x0FE0, 0x07E0, 0x0BE0, 0x03E0, 0x0DE0, 0x05E0, 0x09E0,
0x01E0, 0x00B8, 0x0062, 0x0EE0, 0x06E0, 0x0022, 0x0AE0, 0x02E0,
0x0CE0, 0x04E0, 0x08E0, 0x00E0, 0x0F60, 0x0760, 0x0B60, 0x0360,
0x0D60, 0x0560, 0x1240, 0x0960, 0x0160, 0x0E60, 0x0660, 0x0A60,
0x000F, 0x0250, 0x0038, 0x0260, 0x0050, 0x0C60, 0x0390, 0x00D8,
0x0042, 0x0002, 0x0058, 0x01B0, 0x007C, 0x0029, 0x003C, 0x0098,
0x005C, 0x0009, 0x001C, 0x006C, 0x002C, 0x004C, 0x0018, 0x000C,
0x0074, 0x00E8, 0x0068, 0x0460, 0x0090, 0x0034, 0x00B0, 0x0710,
0x0860, 0x0031, 0x0054, 0x0011, 0x0021, 0x0017, 0x0014, 0x00A8,
0x0028, 0x0001, 0x0310, 0x0130, 0x003E, 0x0064, 0x001E, 0x002E,
0x0024, 0x0510, 0x000E, 0x0036, 0x0016, 0x0044, 0x0030, 0x00C8,
0x01D0, 0x00D0, 0x0110, 0x0048, 0x0610, 0x0150, 0x0060, 0x0088,
0x0FA0, 0x0007, 0x0026, 0x0006, 0x003A, 0x001B, 0x001A, 0x002A,
0x000A, 0x000B, 0x0210, 0x0004, 0x0013, 0x0032, 0x0003, 0x001D,
0x0012, 0x0190, 0x000D, 0x0015, 0x0005, 0x0019, 0x0008, 0x0078,
0x00F0, 0x0070, 0x0290, 0x0410, 0x0010, 0x07A0, 0x0BA0, 0x03A0,
0x0240, 0x1C40, 0x0C40, 0x1440, 0x0440, 0x1840, 0x0840, 0x1040,
0x0040, 0x1F80, 0x0F80, 0x1780, 0x0780, 0x1B80, 0x0B80, 0x1380,
0x0380, 0x1D80, 0x0D80, 0x1580, 0x0580, 0x1980, 0x0980, 0x1180,
0x0180, 0x1E80, 0x0E80, 0x1680, 0x0680, 0x1A80, 0x0A80, 0x1280,
0x0280, 0x1C80, 0x0C80, 0x1480, 0x0480, 0x1880, 0x0880, 0x1080,
0x0080, 0x1F00, 0x0F00, 0x1700, 0x0700, 0x1B00, 0x0B00, 0x1300,
0x0DA0, 0x05A0, 0x09A0, 0x01A0, 0x0EA0, 0x06A0, 0x0AA0, 0x02A0,
0x0CA0, 0x04A0, 0x08A0, 0x00A0, 0x0F20, 0x0720, 0x0B20, 0x0320,
0x0D20, 0x0520, 0x0920, 0x0120, 0x0E20, 0x0620, 0x0A20, 0x0220,
0x0C20, 0x0420, 0x0820, 0x0020, 0x0FC0, 0x07C0, 0x0BC0, 0x03C0,
0x0DC0, 0x05C0, 0x09C0, 0x01C0, 0x0EC0, 0x06C0, 0x0AC0, 0x02C0,
0x0CC0, 0x04C0, 0x08C0, 0x00C0, 0x0F40, 0x0740, 0x0B40, 0x0340,
0x0300, 0x0D40, 0x1D00, 0x0D00, 0x1500, 0x0540, 0x0500, 0x1900,
0x0900, 0x0940, 0x1100, 0x0100, 0x1E00, 0x0E00, 0x0140, 0x1600,
0x0600, 0x1A00, 0x0E40, 0x0640, 0x0A40, 0x0A00, 0x1200, 0x0200,
0x1C00, 0x0C00, 0x1400, 0x0400, 0x1800, 0x0800, 0x1000, 0x0000
};
//-----------------------------------------------------------------------------
// Local variables
static char Copyright[] = "PKWARE Data Compression Library for Win32\r\n"
"Copyright 1989-1995 PKWARE Inc. All Rights Reserved\r\n"
"Patent No. 5,051,745\r\n"
"PKWARE Data Compression Library Reg. U.S. Pat. and Tm. Off.\r\n"
"Version 1.11\r\n";
//-----------------------------------------------------------------------------
// Local functions
// Fills memory block with a character
static void lmemset(void * buff, int c, size_t count)
{
memset(buff, c, count);
}
// Copies memory block to another location
static void lmemcpy(void * trg, const void * src, size_t count)
{
memcpy(trg, src, count);
}
static void SortBuffer(TCmpStruct * pWork, unsigned char * uncmp_data, unsigned char * work_end)
{
unsigned short * pin0DC8;
unsigned char * puncmp;
unsigned long offs1, offs2;
unsigned long ndwords;
unsigned int add;
// Fill 0x480 dwords (0x1200 bytes)
ndwords = (unsigned long)((pWork->out_buff - (char *)pWork->offs0DC8 + 1) >> 2);
if(ndwords <= 1)
ndwords = 1;
memset(pWork->offs0DC8, 0, ndwords << 2);
for(puncmp = uncmp_data; work_end > puncmp; puncmp++)
pWork->offs0DC8[(puncmp[0] * 4) + (puncmp[1] * 5)]++;
add = 0;
for(pin0DC8 = pWork->offs0DC8; pin0DC8 < &pWork->offs1FC8; pin0DC8++)
{
add += *pin0DC8;
*pin0DC8 = (unsigned short)add;
}
for(work_end--; work_end >= uncmp_data; work_end--)
{
offs1 = (work_end[0] * 4) + (work_end[1] * 5); // EAX
offs2 = (unsigned long)(work_end - pWork->work_buff); // EDI
pWork->offs0DC8[offs1]--;
pWork->offs49D0[pWork->offs0DC8[offs1]] = (unsigned short)offs2;
}
}
static void FlushBuf(TCmpStruct * pWork)
{
unsigned char save_ch1;
unsigned char save_ch2;
unsigned int size = 0x800;
pWork->write_buf(pWork->out_buff, &size, pWork->param);
save_ch1 = pWork->out_buff[0x800];
save_ch2 = pWork->out_buff[pWork->out_bytes];
pWork->out_bytes -= 0x800;
lmemset(pWork->out_buff, 0, 0x802);
if(pWork->out_bytes != 0)
pWork->out_buff[0] = save_ch1;
if(pWork->out_bits != 0)
pWork->out_buff[pWork->out_bytes] = save_ch2;
}
static void OutputBits(TCmpStruct * pWork, unsigned int nbits, unsigned long bit_buff)
{
unsigned int out_bits;
// If more than 8 bits to output, do recursion
if(nbits > 8)
{
OutputBits(pWork, 8, bit_buff);
bit_buff >>= 8;
nbits -= 8;
}
// Add bits to the last out byte in out_buff;
out_bits = pWork->out_bits;
pWork->out_buff[pWork->out_bytes] |= (unsigned char)(bit_buff << out_bits);
pWork->out_bits += nbits;
// If 8 or more bits, increment number of bytes
if(pWork->out_bits > 8)
{
pWork->out_bytes++;
bit_buff >>= (8 - out_bits);
pWork->out_buff[pWork->out_bytes] = (unsigned char)bit_buff;
pWork->out_bits &= 7;
}
else
{
pWork->out_bits &= 7;
if(pWork->out_bits == 0)
pWork->out_bytes++;
}
// If there is enough compressed bytes, flush them
if(pWork->out_bytes >= 0x800)
FlushBuf(pWork);
}
static unsigned long FindRep(TCmpStruct * pWork, unsigned char * srcbuff)
{
unsigned short esp12;
unsigned char * esp14;
unsigned short esp18;
unsigned char * srcbuff2;
unsigned char esp20;
unsigned char * srcbuff3;
unsigned short * pin0DC8;
unsigned char * pin27CC;
unsigned short * pin49D0;
unsigned long nreps = 1; // EAX
unsigned long ebx, esi;
unsigned short di;
pin0DC8 = pWork->offs0DC8 + (srcbuff[0] * 4) + (srcbuff[1] * 5);
esi = (unsigned long)(srcbuff - pWork->dsize_bytes - pWork->work_buff + 1);
esp18 = *pin0DC8;
pin49D0 = pWork->offs49D0 + esp18;
if(*pin49D0 < esi)
{
while(*pin49D0 < esi)
{
pin49D0++;
esp18++;
}
*pin0DC8 = esp18;
}
//---------------------------------------------------------------------------
srcbuff2 = srcbuff - 1;
pin49D0 = pWork->offs49D0 + esp18;
pin27CC = pWork->work_buff + *pin49D0;
if(srcbuff2 <= pin27CC)
return 0;
//---------------------------------------------------------------------------
srcbuff3 = srcbuff;
for(;;)
{
if(srcbuff3[nreps-1] == pin27CC[nreps-1] && *srcbuff3 == *pin27CC)
{
//
// The following code does not work when compiled with MSVC.NET 2003
// optimizing compiler. We have to switch the optimizations off to make it work
// I found that in debug version (where the optimizations are off), the value
// of "pin27CC" gets incremented twice (once at below, once in the "for" loop)
//
pin27CC++;
srcbuff3++;
for(ebx = 2; ebx < DICT_OFFSET; ebx++)
{
pin27CC++;
srcbuff3++;
if(*pin27CC != *srcbuff3)
break;
}
srcbuff3 = srcbuff;
if(ebx >= nreps)
{
pWork->offs0000 = (unsigned int)(srcbuff3 - pin27CC + ebx - 1);
if((nreps = ebx) > 10)
break;
}
}
pin49D0++;
esp18++;
pin27CC = pWork->work_buff + *pin49D0;
if(srcbuff2 > pin27CC)
continue;
return (nreps >= 2) ? nreps : 0;
}
//---------------------------------------------------------------------------
if(ebx == DICT_OFFSET)
{
pWork->offs0000--;
return ebx;
}
//---------------------------------------------------------------------------
pin49D0 = pWork->offs49D0 + esp18;
if(pWork->work_buff + pin49D0[1] >= srcbuff2)
return nreps;
//---------------------------------------------------------------------------
di = 0;
pWork->offs09BC[0] = 0xFFFF;
pWork->offs09BC[1] = di;
esp12 = 1;
do
{
esi = di;
if(srcbuff[esp12] != srcbuff[esi])
{
di = pWork->offs09BC[esi];
if(di != 0xFFFF)
continue;
}
pWork->offs09BC[++esp12] = ++di;
}
while(esp12 < nreps);
//---------------------------------------------------------------------------
esi = nreps;
pin27CC = pWork->work_buff + pin49D0[0] + nreps;
esp14 = pin27CC;
for(;;) // 0040268B
{
esi = pWork->offs09BC[esi];
if(esi == 0xFFFF)
esi = 0;
pin49D0 = pWork->offs49D0 + esp18;
do
{
pin49D0++;
esp18++;
pin27CC = pWork->work_buff + pin49D0[0];
if(pin27CC >= srcbuff2)
return nreps;
}
while(pin27CC + esi < esp14);
//---------------------------------------------------------------------------
esp20 = srcbuff[nreps - 2];
if(esp20 == pin27CC[nreps - 2])
{
if(pin27CC + esi != esp14)
{
esp14 = pin27CC;
esi = 0;
}
}
else
{
pin49D0 = pWork->offs49D0 + esp18;
do
{
pin49D0++;
esp18++;
pin27CC = pWork->work_buff + pin49D0[0];
if(pin27CC >= srcbuff2)
return nreps;
}
while(pin27CC[nreps - 2] != esp20 || pin27CC[0] != *srcbuff);
esp14 = pin27CC + 2;
esi = 2;
}
//---------------------------------------------------------------------------
for(; esp14[0] == srcbuff[esi]; esp14++)
{
if(++esi >= DICT_OFFSET)
break;
}
if(esi < nreps)
continue;
pWork->offs0000 = (unsigned int)(srcbuff - pin27CC - 1);
if(esi <= nreps)
continue;
nreps = esi;
if(esi == DICT_OFFSET)
return nreps;
do
{
if(srcbuff[esp12] != srcbuff[di])
{
di = pWork->offs09BC[di];
if(di != 0xFFFF)
continue;
}
pWork->offs09BC[++esp12] = ++di;
}
while(esp12 < esi);
}
}
static void WriteCmpData(TCmpStruct * pWork)
{
unsigned int nreps = 0; // ESP+10 : Number of repeats
unsigned char * uncmp_end; // ESP+14 : End of uncompressed data
unsigned int esp18 = 0; // ESP+18 :
unsigned int bytes_required; // ESP+1C : Number of bytes required to read
unsigned int esp20 = 0; // ESP+20 :
unsigned char * uncmp_begin = pWork->work_buff + UNCMP_OFFSET; // EDI
unsigned long nreps1;
unsigned long save_offs0000 = 0;
// Store the compression type and dictionary size
pWork->out_buff[0] = (char)pWork->ctype;
pWork->out_buff[1] = (char)pWork->dsize_bits;
pWork->out_bytes = 2;
// Reset output buffer to zero
lmemset(&pWork->out_buff[2], 0, sizeof(pWork->out_buff) - 2);
pWork->out_bits = 0;
do
{
int total_loaded = 0;
for(bytes_required = 0x1000; bytes_required != 0; )
{
int loaded = pWork->read_buf((char *)pWork->work_buff + UNCMP_OFFSET + total_loaded,
&bytes_required, pWork->param);
if(loaded == 0)
{
if(total_loaded == 0 && esp20 == 0)
goto __Exit;
esp18 = 1;
break;
}
else
{
total_loaded += loaded;
bytes_required -= loaded;
}
}
uncmp_end = pWork->work_buff + pWork->dsize_bytes + total_loaded;
if(esp18 != 0)
uncmp_end += DICT_OFFSET;
//
// Warning: Passing "uncmp_end + 1" to the SortBuffer function may cause
// the output to be unpredictable in Storm.dll's compression. Because Storm.dll
// does not pass the zeroed buffer to the "implode" function, the byte after
// uncmp_end contains random data. This causes difference within dictionary
// created in SortBuffer function and may also cause different compressed output.
// We always zero the data before compression, so this thing never occurs.
// Funny is that it is actually not a bug, because if we decompress the data back,
// we'll get the identical data with the original input.
//
switch(esp20)
{
case 0:
SortBuffer(pWork, uncmp_begin, uncmp_end + 1);
esp20++;
if(pWork->dsize_bytes != 0x1000)
esp20++;
break;
case 1:
SortBuffer(pWork, uncmp_begin - pWork->dsize_bytes + DICT_OFFSET, uncmp_end + 1);
esp20++;
break;
default:
SortBuffer(pWork, uncmp_begin - pWork->dsize_bytes, uncmp_end + 1);
break;
}
while(uncmp_end > uncmp_begin)
{
nreps1 = FindRep(pWork, uncmp_begin);
while(nreps1 != 0)
{
if(nreps1 == 2 && pWork->offs0000 >= 0x100)
break;
if(esp18 != 0 && uncmp_begin + nreps1 > uncmp_end)
goto _004022DB;
if(nreps1 >= 8 || uncmp_begin + 1 >= uncmp_end)
goto _004022FF;
save_offs0000 = pWork->offs0000; // ebp
nreps = nreps1;
nreps1 = FindRep(pWork, uncmp_begin + 1);
if(nreps >= nreps1)
goto _004022F9;
if(nreps + 1 >= nreps1 && save_offs0000 <= 0x80)
goto _004022F9;
OutputBits(pWork, pWork->nChBits[*uncmp_begin], pWork->nChCodes[*uncmp_begin]);
uncmp_begin++;
}
_0040222F:
OutputBits(pWork, pWork->nChBits[*uncmp_begin], pWork->nChCodes[*uncmp_begin]);
uncmp_begin++;
_00402252:;
}
if(esp18 == 0)
{
uncmp_begin -= 0x1000;
lmemcpy(pWork->work_buff, pWork->work_buff + 0x1000, pWork->dsize_bytes + DICT_OFFSET);
}
}
while(esp18 == 0);
__Exit:
OutputBits(pWork, pWork->nChBits[0x305], pWork->nChCodes[0x305]);
if(pWork->out_bits != 0)
pWork->out_bytes++;
pWork->write_buf(pWork->out_buff, &pWork->out_bytes, pWork->param);
return;
_004022DB:
nreps1 = (unsigned long)(uncmp_end - uncmp_begin);
if(nreps1 < 2)
goto _0040222F;
if(nreps1 != 2 || pWork->offs0000 < 0x100)
goto _004022FF;
goto _0040222F;
_004022F9:
nreps1 = nreps;
pWork->offs0000 = save_offs0000;
_004022FF:
OutputBits(pWork, pWork->nChBits[nreps1 + 0xFE], pWork->nChCodes[nreps1 + 0xFE]);
if(nreps1 == 2)
{
OutputBits(pWork, pWork->dist_bits[pWork->offs0000 >> 2],
pWork->dist_codes[pWork->offs0000 >> 2]);
OutputBits(pWork, 2, pWork->offs0000 & 3);
}
else
{
OutputBits(pWork, pWork->dist_bits[pWork->offs0000 >> pWork->dsize_bits],
pWork->dist_codes[pWork->offs0000 >> pWork->dsize_bits]);
OutputBits(pWork, pWork->dsize_bits, pWork->dsize_mask & pWork->offs0000);
}
uncmp_begin += nreps1;
goto _00402252;
}
//-----------------------------------------------------------------------------
// Main imploding function
unsigned int PKEXPORT implode(
unsigned int (*read_buf)(char *buf, unsigned int *size, void *param),
void (*write_buf)(char *buf, unsigned int *size, void *param),
char *work_buf,
void *param,
unsigned int *type,
unsigned int *dsize)
{
TCmpStruct * pWork = (TCmpStruct *)work_buf;
unsigned int nChCode;
unsigned int nCount;
unsigned int i;
// Initialize the work buffer. This is not in the Pklib,
// but it seems to be a bug. Storm always pre-fills the data with zeros,
// and always compresses one block only. So the bug will not appear.
// But when a larger data block (size > 0x1000) is compressed,
// it may fail.
memset(pWork, 0, sizeof(TCmpStruct));
// Fill the work buffer information
pWork->read_buf = read_buf;
pWork->write_buf = write_buf;
pWork->dsize_bytes = *dsize;
pWork->ctype = *type;
pWork->param = param;
pWork->dsize_bits = 4;
pWork->dsize_mask = 0x0F;
// Test dictionary size
switch(*dsize)
{
case 0x1000 :
pWork->dsize_bits++;
pWork->dsize_mask |= 0x20;
// No break here !!!
case 0x0800 :
pWork->dsize_bits++;
pWork->dsize_mask |= 0x10;
// No break here !!!
case 0x0400 :
break;
default:
return CMP_INVALID_DICTSIZE;
}
// Test the compression type
switch(*type)
{
case CMP_BINARY: // We will compress data with binary compression type
for(nChCode = 0, nCount = 0; nCount < 0x100; nCount++)
{
pWork->nChBits[nCount] = 9;
pWork->nChCodes[nCount] = (unsigned short)nChCode;
nChCode = (nChCode & 0x0000FFFF) + 2;
}
break;
case CMP_ASCII: // We will compress data with ASCII compression type
for(nCount = 0; nCount < 0x100; nCount++)
{
pWork->nChBits[nCount] = (unsigned char )(ChBitsAsc[nCount] + 1);
pWork->nChCodes[nCount] = (unsigned short)(ChCodeAsc[nCount] * 2);
}
break;
default:
return CMP_INVALID_MODE;
}
for(i = 0; i < 0x10; i++)
{
int nCount2 = 0; // EBX
if((1 << ExLenBits[i]) == 0)
continue;
do
{
pWork->nChBits[nCount] = (unsigned char)(ExLenBits[i] + LenBits[i] + 1);
pWork->nChCodes[nCount] = (unsigned short)((nCount2 << (LenBits[i] + 1)) | ((LenCode[i] & 0xFFFF00FF) * 2) | 1);
nCount2++;
nCount++;
}
while((1 << ExLenBits[i]) > nCount2);
}
// Copy the distance codes and distance bits and perform the compression
lmemcpy(&pWork->dist_codes, DistCode, sizeof(DistCode));
lmemcpy(&pWork->dist_bits, DistBits, sizeof(DistBits));
WriteCmpData(pWork);
return CMP_NO_ERROR;
}

137
contrib/vmap_extractor_v2/stormlib/pklib/pklib.h
View file

@ -1,137 +0,0 @@
/*****************************************************************************/
/* pklib.h Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Header file for PKWARE Data Compression Library */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 31.03.03 1.00 Lad The first version of pkware.h */
/*****************************************************************************/
#ifndef __PKLIB_H__
#define __PKLIB_H__
#include "../StormPort.h"
//-----------------------------------------------------------------------------
// Defines
#define CMP_BINARY 0 // Binary compression
#define CMP_ASCII 1 // Ascii compression
#define CMP_NO_ERROR 0
#define CMP_INVALID_DICTSIZE 1
#define CMP_INVALID_MODE 2
#define CMP_BAD_DATA 3
#define CMP_ABORT 4
//-----------------------------------------------------------------------------
// Define calling convention
#ifndef PKEXPORT
#define PKEXPORT //__cdecl // Use for normal __cdecl calling
#endif
//#define PKEXPORT __stdcall
//#define PKEXPORT __fastcall
//-----------------------------------------------------------------------------
// Internal structures
// Compression structure
typedef struct
{
unsigned int offs0000; // 0000 :
unsigned int out_bytes; // 0004 : # bytes available in out_buff
unsigned int out_bits; // 0008 : # of bits available in the last out byte
unsigned int dsize_bits; // 000C : Dict size : 4=0x400, 5=0x800, 6=0x1000
unsigned int dsize_mask; // 0010 : Dict size : 0x0F=0x400, 0x1F=0x800, 0x3F=0x1000
unsigned int ctype; // 0014 : Compression type (Ascii or binary)
unsigned int dsize_bytes; // 0018 : Dictionary size in bytes
unsigned char dist_bits[0x40]; // 001C : Distance bits
unsigned char dist_codes[0x40]; // 005C : Distance codes
unsigned char nChBits[0x306]; // 009C :
unsigned short nChCodes[0x306]; // 03A2 :
unsigned short offs09AE; // 09AE :
void * param; // 09B0 : User parameter
unsigned int (*read_buf)(char *buf, unsigned int *size, void *param); // 9B4
void (*write_buf)(char *buf, unsigned int *size, void *param); // 9B8
unsigned short offs09BC[0x204]; // 09BC :
unsigned long offs0DC4; // 0DC4 :
unsigned short offs0DC8[0x900]; // 0DC8 :
unsigned short offs1FC8; // 1FC8 :
char out_buff[0x802]; // 1FCA : Output (compressed) data
unsigned char work_buff[0x2204]; // 27CC : Work buffer
// + DICT_OFFSET => Dictionary
// + UNCMP_OFFSET => Uncompressed data
unsigned short offs49D0[0x2000]; // 49D0 :
} TCmpStruct;
#define CMP_BUFFER_SIZE sizeof(TCmpStruct) // Size of compression buffer
// Decompression structure
typedef struct
{
unsigned long offs0000; // 0000
unsigned long ctype; // 0004 - Compression type (CMP_BINARY or CMP_ASCII)
unsigned long outputPos; // 0008 - Position in output buffer
unsigned long dsize_bits; // 000C - Dict size (4, 5, 6 for 0x400, 0x800, 0x1000)
unsigned long dsize_mask; // 0010 - Dict size bitmask (0x0F, 0x1F, 0x3F for 0x400, 0x800, 0x1000)
unsigned long bit_buff; // 0014 - 16-bit buffer for processing input data
unsigned long extra_bits; // 0018 - Number of extra (above 8) bits in bit buffer
unsigned int in_pos; // 001C - Position in in_buff
unsigned long in_bytes; // 0020 - Number of bytes in input buffer
void * param; // 0024 - Custom parameter
unsigned int (*read_buf)(char *buf, unsigned int *size, void *param); // 0028
void (*write_buf)(char *buf, unsigned int *size, void *param);// 002C
unsigned char out_buff[0x2000]; // 0030 - Output circle buffer. Starting position is 0x1000
unsigned char offs2030[0x204]; // 2030 - ???
unsigned char in_buff[0x800]; // 2234 - Buffer for data to be decompressed
unsigned char position1[0x100]; // 2A34 - Positions in buffers
unsigned char position2[0x100]; // 2B34 - Positions in buffers
unsigned char offs2C34[0x100]; // 2C34 - Buffer for
unsigned char offs2D34[0x100]; // 2D34 - Buffer for
unsigned char offs2E34[0x80]; // 2EB4 - Buffer for
unsigned char offs2EB4[0x100]; // 2EB4 - Buffer for
unsigned char ChBitsAsc[0x100]; // 2FB4 - Buffer for
unsigned char DistBits[0x40]; // 30B4 - Numbers of bytes to skip copied block length
unsigned char LenBits[0x10]; // 30F4 - Numbers of bits for skip copied block length
unsigned char ExLenBits[0x10]; // 3104 - Number of valid bits for copied block
unsigned short LenBase[0x10]; // 3114 - Buffer for
} TDcmpStruct;
#define EXP_BUFFER_SIZE sizeof(TDcmpStruct) // Size of decompress buffer
//-----------------------------------------------------------------------------
// Public functions
#ifdef __cplusplus
extern "C" {
#endif
unsigned int PKEXPORT implode(
unsigned int (*read_buf)(char *buf, unsigned int *size, void *param),
void (*write_buf)(char *buf, unsigned int *size, void *param),
char *work_buf,
void *param,
unsigned int *type,
unsigned int *dsize);
unsigned int PKEXPORT explode(
unsigned int (*read_buf)(char *buf, unsigned int *size, void *param),
void (*write_buf)(char *buf, unsigned int *size, void *param),
char *work_buf,
void *param);
// The original name "crc32" was changed to "crc32pk" due
// to compatibility with zlib
unsigned long PKEXPORT crc32pk(char *buffer, unsigned int *size, unsigned long *old_crc);
#ifdef __cplusplus
} // End of 'extern "C"' declaration
#endif
#endif // __PKLIB_H__

356
contrib/vmap_extractor_v2/stormlib/wave/wave.cpp
View file

@ -1,356 +0,0 @@
/*****************************************************************************/
/* wave.cpp Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* This module contains decompression methods used by Storm.dll to decompress*/
/* WAVe files. Thanks to Tom Amigo for releasing his sources. */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 11.03.03 1.00 Lad Splitted from Pkware.cpp */
/* 20.05.03 2.00 Lad Added compression */
/* 19.11.03 2.01 Dan Big endian handling */
/*****************************************************************************/
#include "wave.h"
//------------------------------------------------------------------------------
// Structures
union TByteAndWordPtr
{
short * pw;
unsigned char * pb;
};
union TWordAndByteArray
{
short w;
unsigned char b[2];
};
//-----------------------------------------------------------------------------
// Tables necessary dor decompression
static long Table1503F120[] =
{
0xFFFFFFFF, 0x00000000, 0xFFFFFFFF, 0x00000004, 0xFFFFFFFF, 0x00000002, 0xFFFFFFFF, 0x00000006,
0xFFFFFFFF, 0x00000001, 0xFFFFFFFF, 0x00000005, 0xFFFFFFFF, 0x00000003, 0xFFFFFFFF, 0x00000007,
0xFFFFFFFF, 0x00000001, 0xFFFFFFFF, 0x00000005, 0xFFFFFFFF, 0x00000003, 0xFFFFFFFF, 0x00000007,
0xFFFFFFFF, 0x00000002, 0xFFFFFFFF, 0x00000004, 0xFFFFFFFF, 0x00000006, 0xFFFFFFFF, 0x00000008
};
static long Table1503F1A0[] =
{
0x00000007, 0x00000008, 0x00000009, 0x0000000A, 0x0000000B, 0x0000000C, 0x0000000D, 0x0000000E,
0x00000010, 0x00000011, 0x00000013, 0x00000015, 0x00000017, 0x00000019, 0x0000001C, 0x0000001F,
0x00000022, 0x00000025, 0x00000029, 0x0000002D, 0x00000032, 0x00000037, 0x0000003C, 0x00000042,
0x00000049, 0x00000050, 0x00000058, 0x00000061, 0x0000006B, 0x00000076, 0x00000082, 0x0000008F,
0x0000009D, 0x000000AD, 0x000000BE, 0x000000D1, 0x000000E6, 0x000000FD, 0x00000117, 0x00000133,
0x00000151, 0x00000173, 0x00000198, 0x000001C1, 0x000001EE, 0x00000220, 0x00000256, 0x00000292,
0x000002D4, 0x0000031C, 0x0000036C, 0x000003C3, 0x00000424, 0x0000048E, 0x00000502, 0x00000583,
0x00000610, 0x000006AB, 0x00000756, 0x00000812, 0x000008E0, 0x000009C3, 0x00000ABD, 0x00000BD0,
0x00000CFF, 0x00000E4C, 0x00000FBA, 0x0000114C, 0x00001307, 0x000014EE, 0x00001706, 0x00001954,
0x00001BDC, 0x00001EA5, 0x000021B6, 0x00002515, 0x000028CA, 0x00002CDF, 0x0000315B, 0x0000364B,
0x00003BB9, 0x000041B2, 0x00004844, 0x00004F7E, 0x00005771, 0x0000602F, 0x000069CE, 0x00007462,
0x00007FFF
};
//----------------------------------------------------------------------------
// CompressWave
// 1500EF70
int CompressWave(unsigned char * pbOutBuffer, int dwOutLength, short * pwInBuffer, int dwInLength, int nChannels, int nCmpLevel)
// ECX EDX
{
TWordAndByteArray Wcmp;
TByteAndWordPtr out; // Pointer to the output buffer
long SInt32Array1[2];
long SInt32Array2[2];
long SInt32Array3[2];
long nBytesRemains = dwOutLength; // Number of bytes remaining
long nWordsRemains; // Number of words remaining
// unsigned char * pbSaveOutBuffer; // Copy of output buffer (actually not used)
unsigned long dwBitBuff;
unsigned long dwStopBit;
unsigned long dwBit;
unsigned long ebx;
unsigned long esi;
long nTableValue;
long nOneWord;
long var_1C;
long var_2C;
int nLength;
int nIndex;
int nValue;
// If less than 2 bytes remain, don't decompress anything
// pbSaveOutBuffer = pbOutBuffer;
out.pb = pbOutBuffer;
if(nBytesRemains < 2)
return 2;
Wcmp.b[1] = (unsigned char)(nCmpLevel - 1);
Wcmp.b[0] = (unsigned char)0;
*out.pw++ = BSWAP_INT16_SIGNED(Wcmp.w);
if((out.pb - pbOutBuffer + (nChannels * 2)) > nBytesRemains)
return (int)(out.pb - pbOutBuffer + (nChannels * 2));
SInt32Array1[0] = SInt32Array1[1] = 0x2C;
for(int i = 0; i < nChannels; i++)
{
nOneWord = BSWAP_INT16_SIGNED(*pwInBuffer++);
*out.pw++ = BSWAP_INT16_SIGNED((short)nOneWord);
SInt32Array2[i] = nOneWord;
}
// Weird. But it's there
nLength = dwInLength;
if(nLength < 0) // mov eax, dwInLength; cdq; sub eax, edx;
nLength++;
nLength = (nLength / 2) - (int)(out.pb - pbOutBuffer);
nLength = (nLength < 0) ? 0 : nLength;
nIndex = nChannels - 1; // edi
nWordsRemains = dwInLength / 2; // eax
// ebx - nChannels
// ecx - pwOutPos
for(int chnl = nChannels; chnl < nWordsRemains; chnl++)
{
// 1500F030
if((out.pb - pbOutBuffer + 2) > nBytesRemains)
return (int)(out.pb - pbOutBuffer + 2);
// Switch index
if(nChannels == 2)
nIndex = (nIndex == 0) ? 1 : 0;
// Load one word from the input stream
nOneWord = BSWAP_INT16_SIGNED(*pwInBuffer++); // ecx - nOneWord
SInt32Array3[nIndex] = nOneWord;
// esi - SInt32Array2[nIndex]
// eax - nValue
nValue = nOneWord - SInt32Array2[nIndex];
nValue = (nValue < 0) ? ((nValue ^ 0xFFFFFFFF) + 1) : nValue;
ebx = (nOneWord >= SInt32Array2[nIndex]) ? 0 : 0x40;
// esi - SInt32Array2[nIndex]
// edx - Table1503F1A0[SInt32Array2[nIndex]]
// edi - (Table1503F1A0[SInt32Array1[nIndex]] >> nCmpLevel)
nTableValue = Table1503F1A0[SInt32Array1[nIndex]];
dwStopBit = (unsigned long)nCmpLevel;
// edi - nIndex;
if(nValue < (nTableValue >> nCmpLevel))
{
if(SInt32Array1[nIndex] != 0)
SInt32Array1[nIndex]--;
*out.pb++ = 0x80;
}
else
{
while(nValue > nTableValue * 2)
{
if(SInt32Array1[nIndex] >= 0x58 || nLength == 0)
break;
SInt32Array1[nIndex] += 8;
if(SInt32Array1[nIndex] > 0x58)
SInt32Array1[nIndex] = 0x58;
nTableValue = Table1503F1A0[SInt32Array1[nIndex]];
*out.pb++ = 0x81;
nLength--;
}
var_2C = nTableValue >> Wcmp.b[1];
dwBitBuff = 0;
esi = (1 << (dwStopBit - 2));
dwStopBit = (esi <= 0x20) ? esi : 0x20;
for(var_1C = 0, dwBit = 1; ; dwBit <<= 1)
{
// esi = var_1C + nTableValue;
if((var_1C + nTableValue) <= nValue)
{
var_1C += nTableValue;
dwBitBuff |= dwBit;
}
if(dwBit == dwStopBit)
break;
nTableValue >>= 1;
}
nValue = SInt32Array2[nIndex];
if(ebx != 0)
{
nValue -= (var_1C + var_2C);
if(nValue < -32768)
nValue = -32768;
}
else
{
nValue += (var_1C + var_2C);
if(nValue > 32767)
nValue = 32767;
}
SInt32Array2[nIndex] = nValue;
*out.pb++ = (unsigned char)(dwBitBuff | ebx);
nTableValue = Table1503F120[dwBitBuff & 0x1F];
SInt32Array1[nIndex] = SInt32Array1[nIndex] + nTableValue;
if(SInt32Array1[nIndex] < 0)
SInt32Array1[nIndex] = 0;
else if(SInt32Array1[nIndex] > 0x58)
SInt32Array1[nIndex] = 0x58;
}
}
return (int)(out.pb - pbOutBuffer);
}
//----------------------------------------------------------------------------
// DecompressWave
// 1500F230
int DecompressWave(unsigned char * pbOutBuffer, int dwOutLength, unsigned char * pbInBuffer, int dwInLength, int nChannels)
{
TByteAndWordPtr out; // Output buffer
TByteAndWordPtr in;
unsigned char * pbInBufferEnd = (pbInBuffer + dwInLength);
long SInt32Array1[2];
long SInt32Array2[2];
long nOneWord;
int dwOutLengthCopy = dwOutLength;
int nIndex;
SInt32Array1[0] = SInt32Array1[1] = 0x2C;
out.pb = pbOutBuffer;
in.pb = pbInBuffer;
in.pw++;
// Fill the Uint32Array2 array by channel values.
for(int i = 0; i < nChannels; i++)
{
nOneWord = BSWAP_INT16_SIGNED(*in.pw++);
SInt32Array2[i] = nOneWord;
if(dwOutLengthCopy < 2)
return (int)(out.pb - pbOutBuffer);
*out.pw++ = BSWAP_INT16_SIGNED((short)nOneWord);
dwOutLengthCopy -= sizeof(short);
}
// Get the initial index
nIndex = nChannels - 1;
// Perform the decompression
while(in.pb < pbInBufferEnd)
{
unsigned char nOneByte = *in.pb++;
// Switch index
if(nChannels == 2)
nIndex = (nIndex == 0) ? 1 : 0;
// 1500F2A2: Get one byte from input buffer
if(nOneByte & 0x80)
{
switch(nOneByte & 0x7F)
{
case 0: // 1500F315
if(SInt32Array1[nIndex] != 0)
SInt32Array1[nIndex]--;
if(dwOutLengthCopy < 2)
return (int)(out.pb - pbOutBuffer);
*out.pw++ = BSWAP_INT16_SIGNED((unsigned short)SInt32Array2[nIndex]);
dwOutLength -= sizeof(unsigned short);
break;
case 1: // 1500F2E8
SInt32Array1[nIndex] += 8;
if(SInt32Array1[nIndex] > 0x58)
SInt32Array1[nIndex] = 0x58;
if(nChannels == 2)
nIndex = (nIndex == 0) ? 1 : 0;
break;
case 2: // 1500F41E
break;
default: // 1500F2C4
SInt32Array1[nIndex] -= 8;
if(SInt32Array1[nIndex] < 0)
SInt32Array1[nIndex] = 0;
if(nChannels == 2)
nIndex = (nIndex == 0) ? 1 : 0;
break;
}
}
else
{
// 1500F349
long temp1 = Table1503F1A0[SInt32Array1[nIndex]]; // EDI
long temp2 = temp1 >> pbInBuffer[1]; // ESI
long temp3 = SInt32Array2[nIndex]; // ECX
if(nOneByte & 0x01) // EBX = nOneByte
temp2 += (temp1 >> 0);
if(nOneByte & 0x02)
temp2 += (temp1 >> 1);
if(nOneByte & 0x04)
temp2 += (temp1 >> 2);
if(nOneByte & 0x08)
temp2 += (temp1 >> 3);
if(nOneByte & 0x10)
temp2 += (temp1 >> 4);
if(nOneByte & 0x20)
temp2 += (temp1 >> 5);
if(nOneByte & 0x40)
{
temp3 = temp3 - temp2;
if(temp3 <= -32768)
temp3 = -32768;
}
else
{
temp3 = temp3 + temp2;
if(temp3 >= 32767)
temp3 = 32767;
}
SInt32Array2[nIndex] = temp3;
if(dwOutLength < 2)
break;
// Store the output 16-bit value
*out.pw++ = BSWAP_INT16_SIGNED((short)SInt32Array2[nIndex]);
dwOutLength -= 2;
SInt32Array1[nIndex] += Table1503F120[nOneByte & 0x1F];
if(SInt32Array1[nIndex] < 0)
SInt32Array1[nIndex] = 0;
else if(SInt32Array1[nIndex] > 0x58)
SInt32Array1[nIndex] = 0x58;
}
}
return (int)(out.pb - pbOutBuffer);
}

22
contrib/vmap_extractor_v2/stormlib/wave/wave.h
View file

@ -1,22 +0,0 @@
/*****************************************************************************/
/* Wave.h Copyright (c) Ladislav Zezula 2003 */
/*---------------------------------------------------------------------------*/
/* Header file for WAVe unplode functions */
/*---------------------------------------------------------------------------*/
/* Date Ver Who Comment */
/* -------- ---- --- ------- */
/* 31.03.03 1.00 Lad The first version of Wave.h */
/*****************************************************************************/
#ifndef __WAVE_H__
#define __WAVE_H__
//-----------------------------------------------------------------------------
// Functions
#include "../StormPort.h"
int CompressWave (unsigned char * pbOutBuffer, int dwOutLength, short * pwInBuffer, int dwInLength, int nCmpType, int nChannels);
int DecompressWave(unsigned char * pbOutBuffer, int dwOutLength, unsigned char * pbInBuffer, int dwInLength, int nChannels);
#endif // __WAVE_H__

481
contrib/vmap_extractor_v2/stormlib/zlib/ChangeLog
View file

@ -1,481 +0,0 @@
ChangeLog file for zlib
Changes in 1.1.4 (11 March 2002)
- ZFREE was repeated on same allocation on some error conditions.
This creates a security problem described in
http://www.zlib.org/advisory-2002-03-11.txt
- Returned incorrect error (Z_MEM_ERROR) on some invalid data
- Avoid accesses before window for invalid distances with inflate window
less than 32K.
- force windowBits > 8 to avoid a bug in the encoder for a window size
of 256 bytes. (A complete fix will be available in 1.1.5).
Changes in 1.1.3 (9 July 1998)
- fix "an inflate input buffer bug that shows up on rare but persistent
occasions" (Mark)
- fix gzread and gztell for concatenated .gz files (Didier Le Botlan)
- fix gzseek(..., SEEK_SET) in write mode
- fix crc check after a gzeek (Frank Faubert)
- fix miniunzip when the last entry in a zip file is itself a zip file
(J Lillge)
- add contrib/asm586 and contrib/asm686 (Brian Raiter)
See http://www.muppetlabs.com/~breadbox/software/assembly.html
- add support for Delphi 3 in contrib/delphi (Bob Dellaca)
- add support for C++Builder 3 and Delphi 3 in contrib/delphi2 (Davide Moretti)
- do not exit prematurely in untgz if 0 at start of block (Magnus Holmgren)
- use macro EXTERN instead of extern to support DLL for BeOS (Sander Stoks)
- added a FAQ file
- Support gzdopen on Mac with Metrowerks (Jason Linhart)
- Do not redefine Byte on Mac (Brad Pettit & Jason Linhart)
- define SEEK_END too if SEEK_SET is not defined (Albert Chin-A-Young)
- avoid some warnings with Borland C (Tom Tanner)
- fix a problem in contrib/minizip/zip.c for 16-bit MSDOS (Gilles Vollant)
- emulate utime() for WIN32 in contrib/untgz (Gilles Vollant)
- allow several arguments to configure (Tim Mooney, Frodo Looijaard)
- use libdir and includedir in Makefile.in (Tim Mooney)
- support shared libraries on OSF1 V4 (Tim Mooney)
- remove so_locations in "make clean" (Tim Mooney)
- fix maketree.c compilation error (Glenn, Mark)
- Python interface to zlib now in Python 1.5 (Jeremy Hylton)
- new Makefile.riscos (Rich Walker)
- initialize static descriptors in trees.c for embedded targets (Nick Smith)
- use "foo-gz" in example.c for RISCOS and VMS (Nick Smith)
- add the OS/2 files in Makefile.in too (Andrew Zabolotny)
- fix fdopen and halloc macros for Microsoft C 6.0 (Tom Lane)
- fix maketree.c to allow clean compilation of inffixed.h (Mark)
- fix parameter check in deflateCopy (Gunther Nikl)
- cleanup trees.c, use compressed_len only in debug mode (Christian Spieler)
- Many portability patches by Christian Spieler:
. zutil.c, zutil.h: added "const" for zmem*
. Make_vms.com: fixed some typos
. Make_vms.com: msdos/Makefile.*: removed zutil.h from some dependency lists
. msdos/Makefile.msc: remove "default rtl link library" info from obj files
. msdos/Makefile.*: use model-dependent name for the built zlib library
. msdos/Makefile.emx, nt/Makefile.emx, nt/Makefile.gcc:
new makefiles, for emx (DOS/OS2), emx&rsxnt and mingw32 (Windows 9x / NT)
- use define instead of typedef for Bytef also for MSC small/medium (Tom Lane)
- replace __far with _far for better portability (Christian Spieler, Tom Lane)
- fix test for errno.h in configure (Tim Newsham)
Changes in 1.1.2 (19 March 98)
- added contrib/minzip, mini zip and unzip based on zlib (Gilles Vollant)
See http://www.winimage.com/zLibDll/unzip.html
- preinitialize the inflate tables for fixed codes, to make the code
completely thread safe (Mark)
- some simplifications and slight speed-up to the inflate code (Mark)
- fix gzeof on non-compressed files (Allan Schrum)
- add -std1 option in configure for OSF1 to fix gzprintf (Martin Mokrejs)
- use default value of 4K for Z_BUFSIZE for 16-bit MSDOS (Tim Wegner + Glenn)
- added os2/Makefile.def and os2/zlib.def (Andrew Zabolotny)
- add shared lib support for UNIX_SV4.2MP (MATSUURA Takanori)
- do not wrap extern "C" around system includes (Tom Lane)
- mention zlib binding for TCL in README (Andreas Kupries)
- added amiga/Makefile.pup for Amiga powerUP SAS/C PPC (Andreas Kleinert)
- allow "make install prefix=..." even after configure (Glenn Randers-Pehrson)
- allow "configure --prefix $HOME" (Tim Mooney)
- remove warnings in example.c and gzio.c (Glenn Randers-Pehrson)
- move Makefile.sas to amiga/Makefile.sas
Changes in 1.1.1 (27 Feb 98)
- fix macros _tr_tally_* in deflate.h for debug mode (Glenn Randers-Pehrson)
- remove block truncation heuristic which had very marginal effect for zlib
(smaller lit_bufsize than in gzip 1.2.4) and degraded a little the
compression ratio on some files. This also allows inlining _tr_tally for
matches in deflate_slow.
- added msdos/Makefile.w32 for WIN32 Microsoft Visual C++ (Bob Frazier)
Changes in 1.1.0 (24 Feb 98)
- do not return STREAM_END prematurely in inflate (John Bowler)
- revert to the zlib 1.0.8 inflate to avoid the gcc 2.8.0 bug (Jeremy Buhler)
- compile with -DFASTEST to get compression code optimized for speed only
- in minigzip, try mmap'ing the input file first (Miguel Albrecht)
- increase size of I/O buffers in minigzip.c and gzio.c (not a big gain
on Sun but significant on HP)
- add a pointer to experimental unzip library in README (Gilles Vollant)
- initialize variable gcc in configure (Chris Herborth)
Changes in 1.0.9 (17 Feb 1998)
- added gzputs and gzgets functions
- do not clear eof flag in gzseek (Mark Diekhans)
- fix gzseek for files in transparent mode (Mark Diekhans)
- do not assume that vsprintf returns the number of bytes written (Jens Krinke)
- replace EXPORT with ZEXPORT to avoid conflict with other programs
- added compress2 in zconf.h, zlib.def, zlib.dnt
- new asm code from Gilles Vollant in contrib/asm386
- simplify the inflate code (Mark):
. Replace ZALLOC's in huft_build() with single ZALLOC in inflate_blocks_new()
. ZALLOC the length list in inflate_trees_fixed() instead of using stack
. ZALLOC the value area for huft_build() instead of using stack
. Simplify Z_FINISH check in inflate()
- Avoid gcc 2.8.0 comparison bug a little differently than zlib 1.0.8
- in inftrees.c, avoid cc -O bug on HP (Farshid Elahi)
- in zconf.h move the ZLIB_DLL stuff earlier to avoid problems with
the declaration of FAR (Gilles VOllant)
- install libz.so* with mode 755 (executable) instead of 644 (Marc Lehmann)
- read_buf buf parameter of type Bytef* instead of charf*
- zmemcpy parameters are of type Bytef*, not charf* (Joseph Strout)
- do not redeclare unlink in minigzip.c for WIN32 (John Bowler)
- fix check for presence of directories in "make install" (Ian Willis)
Changes in 1.0.8 (27 Jan 1998)
- fixed offsets in contrib/asm386/gvmat32.asm (Gilles Vollant)
- fix gzgetc and gzputc for big endian systems (Markus Oberhumer)
- added compress2() to allow setting the compression level
- include sys/types.h to get off_t on some systems (Marc Lehmann & QingLong)
- use constant arrays for the static trees in trees.c instead of computing
them at run time (thanks to Ken Raeburn for this suggestion). To create
trees.h, compile with GEN_TREES_H and run "make test".
- check return code of example in "make test" and display result
- pass minigzip command line options to file_compress
- simplifying code of inflateSync to avoid gcc 2.8 bug
- support CC="gcc -Wall" in configure -s (QingLong)
- avoid a flush caused by ftell in gzopen for write mode (Ken Raeburn)
- fix test for shared library support to avoid compiler warnings
- zlib.lib -> zlib.dll in msdos/zlib.rc (Gilles Vollant)
- check for TARGET_OS_MAC in addition to MACOS (Brad Pettit)
- do not use fdopen for Metrowerks on Mac (Brad Pettit))
- add checks for gzputc and gzputc in example.c
- avoid warnings in gzio.c and deflate.c (Andreas Kleinert)
- use const for the CRC table (Ken Raeburn)
- fixed "make uninstall" for shared libraries
- use Tracev instead of Trace in infblock.c
- in example.c use correct compressed length for test_sync
- suppress +vnocompatwarnings in configure for HPUX (not always supported)
Changes in 1.0.7 (20 Jan 1998)
- fix gzseek which was broken in write mode
- return error for gzseek to negative absolute position
- fix configure for Linux (Chun-Chung Chen)
- increase stack space for MSC (Tim Wegner)
- get_crc_table and inflateSyncPoint are EXPORTed (Gilles Vollant)
- define EXPORTVA for gzprintf (Gilles Vollant)
- added man page zlib.3 (Rick Rodgers)
- for contrib/untgz, fix makedir() and improve Makefile
- check gzseek in write mode in example.c
- allocate extra buffer for seeks only if gzseek is actually called
- avoid signed/unsigned comparisons (Tim Wegner, Gilles Vollant)
- add inflateSyncPoint in zconf.h
- fix list of exported functions in nt/zlib.dnt and mdsos/zlib.def
Changes in 1.0.6 (19 Jan 1998)
- add functions gzprintf, gzputc, gzgetc, gztell, gzeof, gzseek, gzrewind and
gzsetparams (thanks to Roland Giersig and Kevin Ruland for some of this code)
- Fix a deflate bug occuring only with compression level 0 (thanks to
Andy Buckler for finding this one).
- In minigzip, pass transparently also the first byte for .Z files.
- return Z_BUF_ERROR instead of Z_OK if output buffer full in uncompress()
- check Z_FINISH in inflate (thanks to Marc Schluper)
- Implement deflateCopy (thanks to Adam Costello)
- make static libraries by default in configure, add --shared option.
- move MSDOS or Windows specific files to directory msdos
- suppress the notion of partial flush to simplify the interface
(but the symbol Z_PARTIAL_FLUSH is kept for compatibility with 1.0.4)
- suppress history buffer provided by application to simplify the interface
(this feature was not implemented anyway in 1.0.4)
- next_in and avail_in must be initialized before calling inflateInit or
inflateInit2
- add EXPORT in all exported functions (for Windows DLL)
- added Makefile.nt (thanks to Stephen Williams)
- added the unsupported "contrib" directory:
contrib/asm386/ by Gilles Vollant <info@winimage.com>
386 asm code replacing longest_match().
contrib/iostream/ by Kevin Ruland <kevin@rodin.wustl.edu>
A C++ I/O streams interface to the zlib gz* functions
contrib/iostream2/ by Tyge Løvset <Tyge.Lovset@cmr.no>
Another C++ I/O streams interface
contrib/untgz/ by "Pedro A. Aranda Guti\irrez" <paag@tid.es>
A very simple tar.gz file extractor using zlib
contrib/visual-basic.txt by Carlos Rios <c_rios@sonda.cl>
How to use compress(), uncompress() and the gz* functions from VB.
- pass params -f (filtered data), -h (huffman only), -1 to -9 (compression
level) in minigzip (thanks to Tom Lane)
- use const for rommable constants in deflate
- added test for gzseek and gztell in example.c
- add undocumented function inflateSyncPoint() (hack for Paul Mackerras)
- add undocumented function zError to convert error code to string
(for Tim Smithers)
- Allow compilation of gzio with -DNO_DEFLATE to avoid the compression code.
- Use default memcpy for Symantec MSDOS compiler.
- Add EXPORT keyword for check_func (needed for Windows DLL)
- add current directory to LD_LIBRARY_PATH for "make test"
- create also a link for libz.so.1
- added support for FUJITSU UXP/DS (thanks to Toshiaki Nomura)
- use $(SHAREDLIB) instead of libz.so in Makefile.in (for HPUX)
- added -soname for Linux in configure (Chun-Chung Chen,
- assign numbers to the exported functions in zlib.def (for Windows DLL)
- add advice in zlib.h for best usage of deflateSetDictionary
- work around compiler bug on Atari (cast Z_NULL in call of s->checkfn)
- allow compilation with ANSI keywords only enabled for TurboC in large model
- avoid "versionString"[0] (Borland bug)
- add NEED_DUMMY_RETURN for Borland
- use variable z_verbose for tracing in debug mode (L. Peter Deutsch).
- allow compilation with CC
- defined STDC for OS/2 (David Charlap)
- limit external names to 8 chars for MVS (Thomas Lund)
- in minigzip.c, use static buffers only for 16-bit systems
- fix suffix check for "minigzip -d foo.gz"
- do not return an error for the 2nd of two consecutive gzflush() (Felix Lee)
- use _fdopen instead of fdopen for MSC >= 6.0 (Thomas Fanslau)
- added makelcc.bat for lcc-win32 (Tom St Denis)
- in Makefile.dj2, use copy and del instead of install and rm (Frank Donahoe)
- Avoid expanded $Id$. Use "rcs -kb" or "cvs admin -kb" to avoid Id expansion.
- check for unistd.h in configure (for off_t)
- remove useless check parameter in inflate_blocks_free
- avoid useless assignment of s->check to itself in inflate_blocks_new
- do not flush twice in gzclose (thanks to Ken Raeburn)
- rename FOPEN as F_OPEN to avoid clash with /usr/include/sys/file.h
- use NO_ERRNO_H instead of enumeration of operating systems with errno.h
- work around buggy fclose on pipes for HP/UX
- support zlib DLL with BORLAND C++ 5.0 (thanks to Glenn Randers-Pehrson)
- fix configure if CC is already equal to gcc
Changes in 1.0.5 (3 Jan 98)
- Fix inflate to terminate gracefully when fed corrupted or invalid data
- Use const for rommable constants in inflate
- Eliminate memory leaks on error conditions in inflate
- Removed some vestigial code in inflate
- Update web address in README
Changes in 1.0.4 (24 Jul 96)
- In very rare conditions, deflate(s, Z_FINISH) could fail to produce an EOF
bit, so the decompressor could decompress all the correct data but went
on to attempt decompressing extra garbage data. This affected minigzip too.
- zlibVersion and gzerror return const char* (needed for DLL)
- port to RISCOS (no fdopen, no multiple dots, no unlink, no fileno)
- use z_error only for DEBUG (avoid problem with DLLs)
Changes in 1.0.3 (2 Jul 96)
- use z_streamp instead of z_stream *, which is now a far pointer in MSDOS
small and medium models; this makes the library incompatible with previous
versions for these models. (No effect in large model or on other systems.)
- return OK instead of BUF_ERROR if previous deflate call returned with
avail_out as zero but there is nothing to do
- added memcmp for non STDC compilers
- define NO_DUMMY_DECL for more Mac compilers (.h files merged incorrectly)
- define __32BIT__ if __386__ or i386 is defined (pb. with Watcom and SCO)
- better check for 16-bit mode MSC (avoids problem with Symantec)
Changes in 1.0.2 (23 May 96)
- added Windows DLL support
- added a function zlibVersion (for the DLL support)
- fixed declarations using Bytef in infutil.c (pb with MSDOS medium model)
- Bytef is define's instead of typedef'd only for Borland C
- avoid reading uninitialized memory in example.c
- mention in README that the zlib format is now RFC1950
- updated Makefile.dj2
- added algorithm.doc
Changes in 1.0.1 (20 May 96) [1.0 skipped to avoid confusion]
- fix array overlay in deflate.c which sometimes caused bad compressed data
- fix inflate bug with empty stored block
- fix MSDOS medium model which was broken in 0.99
- fix deflateParams() which could generated bad compressed data.
- Bytef is define'd instead of typedef'ed (work around Borland bug)
- added an INDEX file
- new makefiles for DJGPP (Makefile.dj2), 32-bit Borland (Makefile.b32),
Watcom (Makefile.wat), Amiga SAS/C (Makefile.sas)
- speed up adler32 for modern machines without auto-increment
- added -ansi for IRIX in configure
- static_init_done in trees.c is an int
- define unlink as delete for VMS
- fix configure for QNX
- add configure branch for SCO and HPUX
- avoid many warnings (unused variables, dead assignments, etc...)
- no fdopen for BeOS
- fix the Watcom fix for 32 bit mode (define FAR as empty)
- removed redefinition of Byte for MKWERKS
- work around an MWKERKS bug (incorrect merge of all .h files)
Changes in 0.99 (27 Jan 96)
- allow preset dictionary shared between compressor and decompressor
- allow compression level 0 (no compression)
- add deflateParams in zlib.h: allow dynamic change of compression level
and compression strategy.
- test large buffers and deflateParams in example.c
- add optional "configure" to build zlib as a shared library
- suppress Makefile.qnx, use configure instead
- fixed deflate for 64-bit systems (detected on Cray)
- fixed inflate_blocks for 64-bit systems (detected on Alpha)
- declare Z_DEFLATED in zlib.h (possible parameter for deflateInit2)
- always return Z_BUF_ERROR when deflate() has nothing to do
- deflateInit and inflateInit are now macros to allow version checking
- prefix all global functions and types with z_ with -DZ_PREFIX
- make falloc completely reentrant (inftrees.c)
- fixed very unlikely race condition in ct_static_init
- free in reverse order of allocation to help memory manager
- use zlib-1.0/* instead of zlib/* inside the tar.gz
- make zlib warning-free with "gcc -O3 -Wall -Wwrite-strings -Wpointer-arith
-Wconversion -Wstrict-prototypes -Wmissing-prototypes"
- allow gzread on concatenated .gz files
- deflateEnd now returns Z_DATA_ERROR if it was premature
- deflate is finally (?) fully deterministic (no matches beyond end of input)
- Document Z_SYNC_FLUSH
- add uninstall in Makefile
- Check for __cpluplus in zlib.h
- Better test in ct_align for partial flush
- avoid harmless warnings for Borland C++
- initialize hash_head in deflate.c
- avoid warning on fdopen (gzio.c) for HP cc -Aa
- include stdlib.h for STDC compilers
- include errno.h for Cray
- ignore error if ranlib doesn't exist
- call ranlib twice for NeXTSTEP
- use exec_prefix instead of prefix for libz.a
- renamed ct_* as _tr_* to avoid conflict with applications
- clear z->msg in inflateInit2 before any error return
- initialize opaque in example.c, gzio.c, deflate.c and inflate.c
- fixed typo in zconf.h (_GNUC__ => __GNUC__)
- check for WIN32 in zconf.h and zutil.c (avoid farmalloc in 32-bit mode)
- fix typo in Make_vms.com (f$trnlnm -> f$getsyi)
- in fcalloc, normalize pointer if size > 65520 bytes
- don't use special fcalloc for 32 bit Borland C++
- use STDC instead of __GO32__ to avoid redeclaring exit, calloc, etc...
- use Z_BINARY instead of BINARY
- document that gzclose after gzdopen will close the file
- allow "a" as mode in gzopen.
- fix error checking in gzread
- allow skipping .gz extra-field on pipes
- added reference to Perl interface in README
- put the crc table in FAR data (I dislike more and more the medium model :)
- added get_crc_table
- added a dimension to all arrays (Borland C can't count).
- workaround Borland C bug in declaration of inflate_codes_new & inflate_fast
- guard against multiple inclusion of *.h (for precompiled header on Mac)
- Watcom C pretends to be Microsoft C small model even in 32 bit mode.
- don't use unsized arrays to avoid silly warnings by Visual C++:
warning C4746: 'inflate_mask' : unsized array treated as '__far'
(what's wrong with far data in far model?).
- define enum out of inflate_blocks_state to allow compilation with C++
Changes in 0.95 (16 Aug 95)
- fix MSDOS small and medium model (now easier to adapt to any compiler)
- inlined send_bits
- fix the final (:-) bug for deflate with flush (output was correct but
not completely flushed in rare occasions).
- default window size is same for compression and decompression
(it's now sufficient to set MAX_WBITS in zconf.h).
- voidp -> voidpf and voidnp -> voidp (for consistency with other
typedefs and because voidnp was not near in large model).
Changes in 0.94 (13 Aug 95)
- support MSDOS medium model
- fix deflate with flush (could sometimes generate bad output)
- fix deflateReset (zlib header was incorrectly suppressed)
- added support for VMS
- allow a compression level in gzopen()
- gzflush now calls fflush
- For deflate with flush, flush even if no more input is provided.
- rename libgz.a as libz.a
- avoid complex expression in infcodes.c triggering Turbo C bug
- work around a problem with gcc on Alpha (in INSERT_STRING)
- don't use inline functions (problem with some gcc versions)
- allow renaming of Byte, uInt, etc... with #define.
- avoid warning about (unused) pointer before start of array in deflate.c
- avoid various warnings in gzio.c, example.c, infblock.c, adler32.c, zutil.c
- avoid reserved word 'new' in trees.c
Changes in 0.93 (25 June 95)
- temporarily disable inline functions
- make deflate deterministic
- give enough lookahead for PARTIAL_FLUSH
- Set binary mode for stdin/stdout in minigzip.c for OS/2
- don't even use signed char in inflate (not portable enough)
- fix inflate memory leak for segmented architectures
Changes in 0.92 (3 May 95)
- don't assume that char is signed (problem on SGI)
- Clear bit buffer when starting a stored block
- no memcpy on Pyramid
- suppressed inftest.c
- optimized fill_window, put longest_match inline for gcc
- optimized inflate on stored blocks.
- untabify all sources to simplify patches
Changes in 0.91 (2 May 95)
- Default MEM_LEVEL is 8 (not 9 for Unix) as documented in zlib.h
- Document the memory requirements in zconf.h
- added "make install"
- fix sync search logic in inflateSync
- deflate(Z_FULL_FLUSH) now works even if output buffer too short
- after inflateSync, don't scare people with just "lo world"
- added support for DJGPP
Changes in 0.9 (1 May 95)
- don't assume that zalloc clears the allocated memory (the TurboC bug
was Mark's bug after all :)
- let again gzread copy uncompressed data unchanged (was working in 0.71)
- deflate(Z_FULL_FLUSH), inflateReset and inflateSync are now fully implemented
- added a test of inflateSync in example.c
- moved MAX_WBITS to zconf.h because users might want to change that.
- document explicitly that zalloc(64K) on MSDOS must return a normalized
pointer (zero offset)
- added Makefiles for Microsoft C, Turbo C, Borland C++
- faster crc32()
Changes in 0.8 (29 April 95)
- added fast inflate (inffast.c)
- deflate(Z_FINISH) now returns Z_STREAM_END when done. Warning: this
is incompatible with previous versions of zlib which returned Z_OK.
- work around a TurboC compiler bug (bad code for b << 0, see infutil.h)
(actually that was not a compiler bug, see 0.81 above)
- gzread no longer reads one extra byte in certain cases
- In gzio destroy(), don't reference a freed structure
- avoid many warnings for MSDOS
- avoid the ERROR symbol which is used by MS Windows
Changes in 0.71 (14 April 95)
- Fixed more MSDOS compilation problems :( There is still a bug with
TurboC large model.
Changes in 0.7 (14 April 95)
- Added full inflate support.
- Simplified the crc32() interface. The pre- and post-conditioning
(one's complement) is now done inside crc32(). WARNING: this is
incompatible with previous versions; see zlib.h for the new usage.
Changes in 0.61 (12 April 95)
- workaround for a bug in TurboC. example and minigzip now work on MSDOS.
Changes in 0.6 (11 April 95)
- added minigzip.c
- added gzdopen to reopen a file descriptor as gzFile
- added transparent reading of non-gziped files in gzread.
- fixed bug in gzread (don't read crc as data)
- fixed bug in destroy (gzio.c) (don't return Z_STREAM_END for gzclose).
- don't allocate big arrays in the stack (for MSDOS)
- fix some MSDOS compilation problems
Changes in 0.5:
- do real compression in deflate.c. Z_PARTIAL_FLUSH is supported but
not yet Z_FULL_FLUSH.
- support decompression but only in a single step (forced Z_FINISH)
- added opaque object for zalloc and zfree.
- added deflateReset and inflateReset
- added a variable zlib_version for consistency checking.
- renamed the 'filter' parameter of deflateInit2 as 'strategy'.
Added Z_FILTERED and Z_HUFFMAN_ONLY constants.
Changes in 0.4:
- avoid "zip" everywhere, use zlib instead of ziplib.
- suppress Z_BLOCK_FLUSH, interpret Z_PARTIAL_FLUSH as block flush
if compression method == 8.
- added adler32 and crc32
- renamed deflateOptions as deflateInit2, call one or the other but not both
- added the method parameter for deflateInit2.
- added inflateInit2
- simplied considerably deflateInit and inflateInit by not supporting
user-provided history buffer. This is supported only in deflateInit2
and inflateInit2.
Changes in 0.3:
- prefix all macro names with Z_
- use Z_FINISH instead of deflateEnd to finish compression.
- added Z_HUFFMAN_ONLY
- added gzerror()

100
contrib/vmap_extractor_v2/stormlib/zlib/FAQ
View file

@ -1,100 +0,0 @@
Frequently Asked Questions about zlib
If your question is not there, please check the zlib home page
http://www.zlib.org which may have more recent information.
The lastest zlib FAQ is at http://www.gzip.org/zlib/zlib_faq.html
1. Is zlib Y2K-compliant?
Yes. zlib doesn't handle dates.
2. Where can I get a Windows DLL version?
The zlib sources can be compiled without change to produce a DLL. If you
want a precompiled DLL, see http://www.winimage.com/zLibDll/ . Questions
about the zlib DLL should be sent to Gilles Vollant (info@winimage.com).
3. Where can I get a Visual Basic interface to zlib?
See
* http://www.winimage.com/zLibDll/cmp-z-it.zip
* http://www.dogma.net/markn/articles/zlibtool/zlibtool.htm
* contrib/visual-basic.txt in the zlib distribution
4. compress() returns Z_BUF_ERROR
Make sure that before the call of compress, the length of the compressed
buffer is equal to the total size of the compressed buffer and not
zero. For Visual Basic, check that this parameter is passed by reference
("as any"), not by value ("as long").
5. deflate() or inflate() returns Z_BUF_ERROR
Before making the call, make sure that avail_in and avail_out are not
zero. When setting the parameter flush equal to Z_FINISH, also make sure
that avail_out is big enough to allow processing all pending input.
6. Where's the zlib documentation (man pages, etc.)?
It's in zlib.h for the moment, and Francis S. Lin has converted it to a
web page zlib.html. Volunteers to transform this to Unix-style man pages,
please contact Jean-loup Gailly (jloup@gzip.org). Examples of zlib usage
are in the files example.c and minigzip.c.
7. Why don't you use GNU autoconf or libtool or ...?
Because we would like to keep zlib as a very small and simple
package. zlib is rather portable and doesn't need much configuration.
8. I found a bug in zlib.
Most of the time, such problems are due to an incorrect usage of
zlib. Please try to reproduce the problem with a small program and send
the corresponding source to us at zlib@gzip.org . Do not send
multi-megabyte data files without prior agreement.
9. Why do I get "undefined reference to gzputc"?
If "make test" produces something like
example.o(.text+0x154): undefined reference to `gzputc'
check that you don't have old files libz.* in /usr/lib, /usr/local/lib or
/usr/X11R6/lib. Remove any old versions, then do "make install".
10. I need a Delphi interface to zlib.
See the directories contrib/delphi and contrib/delphi2 in the zlib
distribution.
11. Can zlib handle .zip archives?
See the directory contrib/minizip in the zlib distribution.
12. Can zlib handle .Z files?
No, sorry. You have to spawn an uncompress or gunzip subprocess, or adapt
the code of uncompress on your own.
13. How can I make a Unix shared library?
make clean
./configure -s
make
14. Why does "make test" fail on Mac OS X?
Mac OS X already includes zlib as a shared library, and so -lz links the
shared library instead of the one that the "make" compiled. For zlib
1.1.3, the two are incompatible due to different compile-time
options. Simply change the -lz in the Makefile to libz.a, and it will use
the compiled library instead of the shared one and the "make test" will
succeed.
15. I have a question about OttoPDF
We are not the authors of OttoPDF. The real author is on the OttoPDF web
site Joel Hainley jhainley@myndkryme.com.

86
contrib/vmap_extractor_v2/stormlib/zlib/INDEX
View file

@ -1,86 +0,0 @@
ChangeLog history of changes
INDEX this file
FAQ Frequently Asked Questions about zlib
Make_vms.com script for Vax/VMS
Makefile makefile for Unix (generated by configure)
Makefile.in makefile for Unix (template for configure)
Makefile.riscos makefile for RISCOS
README guess what
algorithm.txt description of the (de)compression algorithm
configure configure script for Unix
descrip.mms makefile for Vax/VMS
zlib.3 mini man page for zlib (volunteers to write full
man pages from zlib.h welcome. write to jloup@gzip.org)
amiga/Makefile.sas makefile for Amiga SAS/C
amiga/Makefile.pup makefile for Amiga powerUP SAS/C PPC
msdos/Makefile.w32 makefile for Microsoft Visual C++ 32-bit
msdos/Makefile.b32 makefile for Borland C++ 32-bit
msdos/Makefile.bor makefile for Borland C/C++ 16-bit
msdos/Makefile.dj2 makefile for DJGPP 2.x
msdos/Makefile.emx makefile for EMX 0.9c (32-bit DOS/OS2)
msdos/Makefile.msc makefile for Microsoft C 16-bit
msdos/Makefile.tc makefile for Turbo C
msdos/Makefile.wat makefile for Watcom C
msdos/zlib.def definition file for Windows DLL
msdos/zlib.rc definition file for Windows DLL
nt/Makefile.nt makefile for Windows NT
nt/zlib.dnt definition file for Windows NT DLL
nt/Makefile.emx makefile for EMX 0.9c/RSXNT 1.41 (Win32 Intel)
nt/Makefile.gcc makefile for Windows NT using GCC (mingw32)
zlib public header files (must be kept):
zconf.h
zlib.h
private source files used to build the zlib library:
adler32.c
compress.c
crc32.c
deflate.c
deflate.h
gzio.c
infblock.c
infblock.h
infcodes.c
infcodes.h
inffast.c
inffast.h
inflate.c
inftrees.c
inftrees.h
infutil.c
infutil.h
maketree.c
trees.c
uncompr.c
zutil.c
zutil.h
source files for sample programs:
example.c
minigzip.c
unsupported contribution by third parties
contrib/asm386/ by Gilles Vollant <info@winimage.com>
386 asm code replacing longest_match().
contrib/minizip/ by Gilles Vollant <info@winimage.com>
Mini zip and unzip based on zlib
See http://www.winimage.com/zLibDll/unzip.html
contrib/iostream/ by Kevin Ruland <kevin@rodin.wustl.edu>
A C++ I/O streams interface to the zlib gz* functions
contrib/iostream2/ by Tyge Løvset <Tyge.Lovset@cmr.no>
Another C++ I/O streams interface
contrib/untgz/ by "Pedro A. Aranda Guti\irrez" <paag@tid.es>
A very simple tar.gz extractor using zlib
contrib/visual-basic.txt by Carlos Rios <c_rios@sonda.cl>
How to use compress(), uncompress() and the gz* functions from VB.

151
contrib/vmap_extractor_v2/stormlib/zlib/Makefile.riscos
View file

@ -1,151 +0,0 @@
# Project: zlib_1_03
# Patched for zlib 1.1.2 rw@shadow.org.uk 19980430
# test works out-of-the-box, installs `somewhere' on demand
# Toolflags:
CCflags = -c -depend !Depend -IC: -g -throwback -DRISCOS -fah
C++flags = -c -depend !Depend -IC: -throwback
Linkflags = -aif -c++ -o $@
ObjAsmflags = -throwback -NoCache -depend !Depend
CMHGflags =
LibFileflags = -c -l -o $@
Squeezeflags = -o $@
# change the line below to where _you_ want the library installed.
libdest = lib:zlib
# Final targets:
@.lib: @.o.adler32 @.o.compress @.o.crc32 @.o.deflate @.o.gzio \
@.o.infblock @.o.infcodes @.o.inffast @.o.inflate @.o.inftrees @.o.infutil @.o.trees \
@.o.uncompr @.o.zutil
LibFile $(LibFileflags) @.o.adler32 @.o.compress @.o.crc32 @.o.deflate \
@.o.gzio @.o.infblock @.o.infcodes @.o.inffast @.o.inflate @.o.inftrees @.o.infutil \
@.o.trees @.o.uncompr @.o.zutil
test: @.minigzip @.example @.lib
@copy @.lib @.libc A~C~DF~L~N~P~Q~RS~TV
@echo running tests: hang on.
@/@.minigzip -f -9 libc
@/@.minigzip -d libc-gz
@/@.minigzip -f -1 libc
@/@.minigzip -d libc-gz
@/@.minigzip -h -9 libc
@/@.minigzip -d libc-gz
@/@.minigzip -h -1 libc
@/@.minigzip -d libc-gz
@/@.minigzip -9 libc
@/@.minigzip -d libc-gz
@/@.minigzip -1 libc
@/@.minigzip -d libc-gz
@diff @.lib @.libc
@echo that should have reported '@.lib and @.libc identical' if you have diff.
@/@.example @.fred @.fred
@echo that will have given lots of hello!'s.
@.minigzip: @.o.minigzip @.lib C:o.Stubs
Link $(Linkflags) @.o.minigzip @.lib C:o.Stubs
@.example: @.o.example @.lib C:o.Stubs
Link $(Linkflags) @.o.example @.lib C:o.Stubs
install: @.lib
cdir $(libdest)
cdir $(libdest).h
@copy @.h.zlib $(libdest).h.zlib A~C~DF~L~N~P~Q~RS~TV
@copy @.h.zconf $(libdest).h.zconf A~C~DF~L~N~P~Q~RS~TV
@copy @.lib $(libdest).lib A~C~DF~L~N~P~Q~RS~TV
@echo okay, installed zlib in $(libdest)
clean:; remove @.minigzip
remove @.example
remove @.libc
-wipe @.o.* F~r~cV
remove @.fred
# User-editable dependencies:
.c.o:
cc $(ccflags) -o $@ $<
# Static dependencies:
# Dynamic dependencies:
o.example: c.example
o.example: h.zlib
o.example: h.zconf
o.minigzip: c.minigzip
o.minigzip: h.zlib
o.minigzip: h.zconf
o.adler32: c.adler32
o.adler32: h.zlib
o.adler32: h.zconf
o.compress: c.compress
o.compress: h.zlib
o.compress: h.zconf
o.crc32: c.crc32
o.crc32: h.zlib
o.crc32: h.zconf
o.deflate: c.deflate
o.deflate: h.deflate
o.deflate: h.zutil
o.deflate: h.zlib
o.deflate: h.zconf
o.gzio: c.gzio
o.gzio: h.zutil
o.gzio: h.zlib
o.gzio: h.zconf
o.infblock: c.infblock
o.infblock: h.zutil
o.infblock: h.zlib
o.infblock: h.zconf
o.infblock: h.infblock
o.infblock: h.inftrees
o.infblock: h.infcodes
o.infblock: h.infutil
o.infcodes: c.infcodes
o.infcodes: h.zutil
o.infcodes: h.zlib
o.infcodes: h.zconf
o.infcodes: h.inftrees
o.infcodes: h.infblock
o.infcodes: h.infcodes
o.infcodes: h.infutil
o.infcodes: h.inffast
o.inffast: c.inffast
o.inffast: h.zutil
o.inffast: h.zlib
o.inffast: h.zconf
o.inffast: h.inftrees
o.inffast: h.infblock
o.inffast: h.infcodes
o.inffast: h.infutil
o.inffast: h.inffast
o.inflate: c.inflate
o.inflate: h.zutil
o.inflate: h.zlib
o.inflate: h.zconf
o.inflate: h.infblock
o.inftrees: c.inftrees
o.inftrees: h.zutil
o.inftrees: h.zlib
o.inftrees: h.zconf
o.inftrees: h.inftrees
o.inftrees: h.inffixed
o.infutil: c.infutil
o.infutil: h.zutil
o.infutil: h.zlib
o.infutil: h.zconf
o.infutil: h.infblock
o.infutil: h.inftrees
o.infutil: h.infcodes
o.infutil: h.infutil
o.trees: c.trees
o.trees: h.deflate
o.trees: h.zutil
o.trees: h.zlib
o.trees: h.zconf
o.trees: h.trees
o.uncompr: c.uncompr
o.uncompr: h.zlib
o.uncompr: h.zconf
o.zutil: c.zutil
o.zutil: h.zutil
o.zutil: h.zlib
o.zutil: h.zconf

147
contrib/vmap_extractor_v2/stormlib/zlib/README
View file

@ -1,147 +0,0 @@
zlib 1.1.4 is a general purpose data compression library. All the code
is thread safe. The data format used by the zlib library
is described by RFCs (Request for Comments) 1950 to 1952 in the files
http://www.ietf.org/rfc/rfc1950.txt (zlib format), rfc1951.txt (deflate
format) and rfc1952.txt (gzip format). These documents are also available in
other formats from ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html
All functions of the compression library are documented in the file zlib.h
(volunteer to write man pages welcome, contact jloup@gzip.org). A usage
example of the library is given in the file example.c which also tests that
the library is working correctly. Another example is given in the file
minigzip.c. The compression library itself is composed of all source files
except example.c and minigzip.c.
To compile all files and run the test program, follow the instructions
given at the top of Makefile. In short "make test; make install"
should work for most machines. For Unix: "./configure; make test; make install"
For MSDOS, use one of the special makefiles such as Makefile.msc.
For VMS, use Make_vms.com or descrip.mms.
Questions about zlib should be sent to <zlib@gzip.org>, or to
Gilles Vollant <info@winimage.com> for the Windows DLL version.
The zlib home page is http://www.zlib.org or http://www.gzip.org/zlib/
Before reporting a problem, please check this site to verify that
you have the latest version of zlib; otherwise get the latest version and
check whether the problem still exists or not.
PLEASE read the zlib FAQ http://www.gzip.org/zlib/zlib_faq.html
before asking for help.
Mark Nelson <markn@ieee.org> wrote an article about zlib for the Jan. 1997
issue of Dr. Dobb's Journal; a copy of the article is available in
http://dogma.net/markn/articles/zlibtool/zlibtool.htm
The changes made in version 1.1.4 are documented in the file ChangeLog.
The only changes made since 1.1.3 are bug corrections:
- ZFREE was repeated on same allocation on some error conditions.
This creates a security problem described in
http://www.zlib.org/advisory-2002-03-11.txt
- Returned incorrect error (Z_MEM_ERROR) on some invalid data
- Avoid accesses before window for invalid distances with inflate window
less than 32K.
- force windowBits > 8 to avoid a bug in the encoder for a window size
of 256 bytes. (A complete fix will be available in 1.1.5).
The beta version 1.1.5beta includes many more changes. A new official
version 1.1.5 will be released as soon as extensive testing has been
completed on it.
Unsupported third party contributions are provided in directory "contrib".
A Java implementation of zlib is available in the Java Development Kit
http://www.javasoft.com/products/JDK/1.1/docs/api/Package-java.util.zip.html
See the zlib home page http://www.zlib.org for details.
A Perl interface to zlib written by Paul Marquess <pmarquess@bfsec.bt.co.uk>
is in the CPAN (Comprehensive Perl Archive Network) sites
http://www.cpan.org/modules/by-module/Compress/
A Python interface to zlib written by A.M. Kuchling <amk@magnet.com>
is available in Python 1.5 and later versions, see
http://www.python.org/doc/lib/module-zlib.html
A zlib binding for TCL written by Andreas Kupries <a.kupries@westend.com>
is availlable at http://www.westend.com/~kupries/doc/trf/man/man.html
An experimental package to read and write files in .zip format,
written on top of zlib by Gilles Vollant <info@winimage.com>, is
available at http://www.winimage.com/zLibDll/unzip.html
and also in the contrib/minizip directory of zlib.
Notes for some targets:
- To build a Windows DLL version, include in a DLL project zlib.def, zlib.rc
and all .c files except example.c and minigzip.c; compile with -DZLIB_DLL
The zlib DLL support was initially done by Alessandro Iacopetti and is
now maintained by Gilles Vollant <info@winimage.com>. Check the zlib DLL
home page at http://www.winimage.com/zLibDll
From Visual Basic, you can call the DLL functions which do not take
a structure as argument: compress, uncompress and all gz* functions.
See contrib/visual-basic.txt for more information, or get
http://www.tcfb.com/dowseware/cmp-z-it.zip
- For 64-bit Irix, deflate.c must be compiled without any optimization.
With -O, one libpng test fails. The test works in 32 bit mode (with
the -n32 compiler flag). The compiler bug has been reported to SGI.
- zlib doesn't work with gcc 2.6.3 on a DEC 3000/300LX under OSF/1 2.1
it works when compiled with cc.
- on Digital Unix 4.0D (formely OSF/1) on AlphaServer, the cc option -std1
is necessary to get gzprintf working correctly. This is done by configure.
- zlib doesn't work on HP-UX 9.05 with some versions of /bin/cc. It works
with other compilers. Use "make test" to check your compiler.
- gzdopen is not supported on RISCOS, BEOS and by some Mac compilers.
- For Turbo C the small model is supported only with reduced performance to
avoid any far allocation; it was tested with -DMAX_WBITS=11 -DMAX_MEM_LEVEL=3
- For PalmOs, see http://www.cs.uit.no/~perm/PASTA/pilot/software.html
Per Harald Myrvang <perm@stud.cs.uit.no>
Acknowledgments:
The deflate format used by zlib was defined by Phil Katz. The deflate
and zlib specifications were written by L. Peter Deutsch. Thanks to all the
people who reported problems and suggested various improvements in zlib;
they are too numerous to cite here.
Copyright notice:
(C) 1995-2002 Jean-loup Gailly and Mark Adler
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
Jean-loup Gailly Mark Adler
jloup@gzip.org madler@alumni.caltech.edu
If you use the zlib library in a product, we would appreciate *not*
receiving lengthy legal documents to sign. The sources are provided
for free but without warranty of any kind. The library has been
entirely written by Jean-loup Gailly and Mark Adler; it does not
include third-party code.
If you redistribute modified sources, we would appreciate that you include
in the file ChangeLog history information documenting your changes.

48
contrib/vmap_extractor_v2/stormlib/zlib/adler32.c
View file

@ -1,48 +0,0 @@
/* adler32.c -- compute the Adler-32 checksum of a data stream
* Copyright (C) 1995-2002 Mark Adler
* For conditions of distribution and use, see copyright notice in zlib.h
*/
/* @(#) $Id$ */
#include "zlib.h"
#define BASE 65521L /* largest prime smaller than 65536 */
#define NMAX 5552
/* NMAX is the largest n such that 255n(n+1)/2 + (n+1)(BASE-1) <= 2^32-1 */
#define DO1(buf,i) {s1 += buf[i]; s2 += s1;}
#define DO2(buf,i) DO1(buf,i); DO1(buf,i+1);
#define DO4(buf,i) DO2(buf,i); DO2(buf,i+2);
#define DO8(buf,i) DO4(buf,i); DO4(buf,i+4);
#define DO16(buf) DO8(buf,0); DO8(buf,8);
/* ========================================================================= */
uLong ZEXPORT adler32(adler, buf, len)
uLong adler;
const Bytef *buf;
uInt len;
{
unsigned long s1 = adler & 0xffff;
unsigned long s2 = (adler >> 16) & 0xffff;
int k;
if (buf == Z_NULL) return 1L;
while (len > 0) {
k = len < NMAX ? len : NMAX;
len -= k;
while (k >= 16) {
DO16(buf);
buf += 16;
k -= 16;
}
if (k != 0) do {
s1 += *buf++;
s2 += s1;
} while (--k);
s1 %= BASE;
s2 %= BASE;
}
return (s2 << 16) | s1;
}

213
contrib/vmap_extractor_v2/stormlib/zlib/algorithm.txt
View file

@ -1,213 +0,0 @@
1. Compression algorithm (deflate)
The deflation algorithm used by gzip (also zip and zlib) is a variation of
LZ77 (Lempel-Ziv 1977, see reference below). It finds duplicated strings in
the input data. The second occurrence of a string is replaced by a
pointer to the previous string, in the form of a pair (distance,
length). Distances are limited to 32K bytes, and lengths are limited
to 258 bytes. When a string does not occur anywhere in the previous
32K bytes, it is emitted as a sequence of literal bytes. (In this
description, `string' must be taken as an arbitrary sequence of bytes,
and is not restricted to printable characters.)
Literals or match lengths are compressed with one Huffman tree, and
match distances are compressed with another tree. The trees are stored
in a compact form at the start of each block. The blocks can have any
size (except that the compressed data for one block must fit in
available memory). A block is terminated when deflate() determines that
it would be useful to start another block with fresh trees. (This is
somewhat similar to the behavior of LZW-based _compress_.)
Duplicated strings are found using a hash table. All input strings of
length 3 are inserted in the hash table. A hash index is computed for
the next 3 bytes. If the hash chain for this index is not empty, all
strings in the chain are compared with the current input string, and
the longest match is selected.
The hash chains are searched starting with the most recent strings, to
favor small distances and thus take advantage of the Huffman encoding.
The hash chains are singly linked. There are no deletions from the
hash chains, the algorithm simply discards matches that are too old.
To avoid a worst-case situation, very long hash chains are arbitrarily
truncated at a certain length, determined by a runtime option (level
parameter of deflateInit). So deflate() does not always find the longest
possible match but generally finds a match which is long enough.
deflate() also defers the selection of matches with a lazy evaluation
mechanism. After a match of length N has been found, deflate() searches for
a longer match at the next input byte. If a longer match is found, the
previous match is truncated to a length of one (thus producing a single
literal byte) and the process of lazy evaluation begins again. Otherwise,
the original match is kept, and the next match search is attempted only N
steps later.
The lazy match evaluation is also subject to a runtime parameter. If
the current match is long enough, deflate() reduces the search for a longer
match, thus speeding up the whole process. If compression ratio is more
important than speed, deflate() attempts a complete second search even if
the first match is already long enough.
The lazy match evaluation is not performed for the fastest compression
modes (level parameter 1 to 3). For these fast modes, new strings
are inserted in the hash table only when no match was found, or
when the match is not too long. This degrades the compression ratio
but saves time since there are both fewer insertions and fewer searches.
2. Decompression algorithm (inflate)
2.1 Introduction
The real question is, given a Huffman tree, how to decode fast. The most
important realization is that shorter codes are much more common than
longer codes, so pay attention to decoding the short codes fast, and let
the long codes take longer to decode.
inflate() sets up a first level table that covers some number of bits of
input less than the length of longest code. It gets that many bits from the
stream, and looks it up in the table. The table will tell if the next
code is that many bits or less and how many, and if it is, it will tell
the value, else it will point to the next level table for which inflate()
grabs more bits and tries to decode a longer code.
How many bits to make the first lookup is a tradeoff between the time it
takes to decode and the time it takes to build the table. If building the
table took no time (and if you had infinite memory), then there would only
be a first level table to cover all the way to the longest code. However,
building the table ends up taking a lot longer for more bits since short
codes are replicated many times in such a table. What inflate() does is
simply to make the number of bits in the first table a variable, and set it
for the maximum speed.
inflate() sends new trees relatively often, so it is possibly set for a
smaller first level table than an application that has only one tree for
all the data. For inflate, which has 286 possible codes for the
literal/length tree, the size of the first table is nine bits. Also the
distance trees have 30 possible values, and the size of the first table is
six bits. Note that for each of those cases, the table ended up one bit
longer than the ``average'' code length, i.e. the code length of an
approximately flat code which would be a little more than eight bits for
286 symbols and a little less than five bits for 30 symbols. It would be
interesting to see if optimizing the first level table for other
applications gave values within a bit or two of the flat code size.
2.2 More details on the inflate table lookup
Ok, you want to know what this cleverly obfuscated inflate tree actually
looks like. You are correct that it's not a Huffman tree. It is simply a
lookup table for the first, let's say, nine bits of a Huffman symbol. The
symbol could be as short as one bit or as long as 15 bits. If a particular
symbol is shorter than nine bits, then that symbol's translation is duplicated
in all those entries that start with that symbol's bits. For example, if the
symbol is four bits, then it's duplicated 32 times in a nine-bit table. If a
symbol is nine bits long, it appears in the table once.
If the symbol is longer than nine bits, then that entry in the table points
to another similar table for the remaining bits. Again, there are duplicated
entries as needed. The idea is that most of the time the symbol will be short
and there will only be one table look up. (That's whole idea behind data
compression in the first place.) For the less frequent long symbols, there
will be two lookups. If you had a compression method with really long
symbols, you could have as many levels of lookups as is efficient. For
inflate, two is enough.
So a table entry either points to another table (in which case nine bits in
the above example are gobbled), or it contains the translation for the symbol
and the number of bits to gobble. Then you start again with the next
ungobbled bit.
You may wonder: why not just have one lookup table for how ever many bits the
longest symbol is? The reason is that if you do that, you end up spending
more time filling in duplicate symbol entries than you do actually decoding.
At least for deflate's output that generates new trees every several 10's of
kbytes. You can imagine that filling in a 2^15 entry table for a 15-bit code
would take too long if you're only decoding several thousand symbols. At the
other extreme, you could make a new table for every bit in the code. In fact,
that's essentially a Huffman tree. But then you spend two much time
traversing the tree while decoding, even for short symbols.
So the number of bits for the first lookup table is a trade of the time to
fill out the table vs. the time spent looking at the second level and above of
the table.
Here is an example, scaled down:
The code being decoded, with 10 symbols, from 1 to 6 bits long:
A: 0
B: 10
C: 1100
D: 11010
E: 11011
F: 11100
G: 11101
H: 11110
I: 111110
J: 111111
Let's make the first table three bits long (eight entries):
000: A,1
001: A,1
010: A,1
011: A,1
100: B,2
101: B,2
110: -> table X (gobble 3 bits)
111: -> table Y (gobble 3 bits)
Each entry is what the bits decode to and how many bits that is, i.e. how
many bits to gobble. Or the entry points to another table, with the number of
bits to gobble implicit in the size of the table.
Table X is two bits long since the longest code starting with 110 is five bits
long:
00: C,1
01: C,1
10: D,2
11: E,2
Table Y is three bits long since the longest code starting with 111 is six
bits long:
000: F,2
001: F,2
010: G,2
011: G,2
100: H,2
101: H,2
110: I,3
111: J,3
So what we have here are three tables with a total of 20 entries that had to
be constructed. That's compared to 64 entries for a single table. Or
compared to 16 entries for a Huffman tree (six two entry tables and one four
entry table). Assuming that the code ideally represents the probability of
the symbols, it takes on the average 1.25 lookups per symbol. That's compared
to one lookup for the single table, or 1.66 lookups per symbol for the
Huffman tree.
There, I think that gives you a picture of what's going on. For inflate, the
meaning of a particular symbol is often more than just a letter. It can be a
byte (a "literal"), or it can be either a length or a distance which
indicates a base value and a number of bits to fetch after the code that is
added to the base value. Or it might be the special end-of-block code. The
data structures created in inftrees.c try to encode all that information
compactly in the tables.
Jean-loup Gailly Mark Adler
jloup@gzip.org madler@alumni.caltech.edu
References:
[LZ77] Ziv J., Lempel A., ``A Universal Algorithm for Sequential Data
Compression,'' IEEE Transactions on Information Theory, Vol. 23, No. 3,
pp. 337-343.
``DEFLATE Compressed Data Format Specification'' available in
ftp://ds.internic.net/rfc/rfc1951.txt

66
contrib/vmap_extractor_v2/stormlib/zlib/amiga/Makefile.pup
View file

@ -1,66 +0,0 @@
# Amiga powerUP (TM) Makefile
# makefile for libpng and SAS C V6.58/7.00 PPC compiler
# Copyright (C) 1998 by Andreas R. Kleinert
CC = scppc
CFLAGS = NOSTKCHK NOSINT OPTIMIZE OPTGO OPTPEEP OPTINLOCAL OPTINL \
OPTLOOP OPTRDEP=8 OPTDEP=8 OPTCOMP=8
LIBNAME = libzip.a
AR = ppc-amigaos-ar
AR_FLAGS = cr
RANLIB = ppc-amigaos-ranlib
LDFLAGS = -r -o
LDLIBS = LIB:scppc.a
LN = ppc-amigaos-ld
RM = delete quiet
OBJS = adler32.o compress.o crc32.o gzio.o uncompr.o deflate.o trees.o \
zutil.o inflate.o infblock.o inftrees.o infcodes.o infutil.o inffast.o
TEST_OBJS = example.o minigzip.o
all: example minigzip
test: all
example
echo hello world | minigzip | minigzip -d
$(LIBNAME): $(OBJS)
$(AR) $(AR_FLAGS) $@ $(OBJS)
$(RANLIB) $@
example: example.o $(LIBNAME)
$(LN) $(LDFLAGS) example LIB:c_ppc.o example.o $(LIBNAME) $(LDLIBS) LIB:end.o
minigzip: minigzip.o $(LIBNAME)
$(LN) $(LDFLAGS) minigzip LIB:c_ppc.o minigzip.o $(LIBNAME) $(LDLIBS) LIB:end.o
clean:
$(RM) *.o example minigzip $(LIBNAME) foo.gz
zip:
zip -ul9 zlib README ChangeLog Makefile Make????.??? Makefile.?? \
descrip.mms *.[ch]
tgz:
cd ..; tar cfz zlib/zlib.tgz zlib/README zlib/ChangeLog zlib/Makefile \
zlib/Make????.??? zlib/Makefile.?? zlib/descrip.mms zlib/*.[ch]
# DO NOT DELETE THIS LINE -- make depend depends on it.
adler32.o: zutil.h zlib.h zconf.h
compress.o: zlib.h zconf.h
crc32.o: zutil.h zlib.h zconf.h
deflate.o: deflate.h zutil.h zlib.h zconf.h
example.o: zlib.h zconf.h
gzio.o: zutil.h zlib.h zconf.h
infblock.o: zutil.h zlib.h zconf.h infblock.h inftrees.h infcodes.h infutil.h
infcodes.o: zutil.h zlib.h zconf.h inftrees.h infutil.h infcodes.h inffast.h
inffast.o: zutil.h zlib.h zconf.h inftrees.h infutil.h inffast.h
inflate.o: zutil.h zlib.h zconf.h infblock.h
inftrees.o: zutil.h zlib.h zconf.h inftrees.h
infutil.o: zutil.h zlib.h zconf.h inftrees.h infutil.h
minigzip.o: zlib.h zconf.h
trees.o: deflate.h zutil.h zlib.h zconf.h
uncompr.o: zlib.h zconf.h
zutil.o: zutil.h zlib.h zconf.h

64
contrib/vmap_extractor_v2/stormlib/zlib/amiga/Makefile.sas
View file

@ -1,64 +0,0 @@
# SMakefile for zlib
# Modified from the standard UNIX Makefile Copyright Jean-loup Gailly
# Osma Ahvenlampi <Osma.Ahvenlampi@hut.fi>
# Amiga, SAS/C 6.56 & Smake
CC=sc
CFLAGS=OPT
#CFLAGS=OPT CPU=68030
#CFLAGS=DEBUG=LINE
LDFLAGS=LIB z.lib
SCOPTIONS=OPTSCHED OPTINLINE OPTALIAS OPTTIME OPTINLOCAL STRMERGE \
NOICONS PARMS=BOTH NOSTACKCHECK UTILLIB NOVERSION ERRORREXX
OBJS = adler32.o compress.o crc32.o gzio.o uncompr.o deflate.o trees.o \
zutil.o inflate.o infblock.o inftrees.o infcodes.o infutil.o inffast.o
TEST_OBJS = example.o minigzip.o
all: SCOPTIONS example minigzip
test: all
`cd`/example
echo hello world | minigzip | minigzip -d
install: z.lib
copy zlib.h zconf.h INCLUDE: clone
copy z.lib LIB: clone
z.lib: $(OBJS)
oml z.lib r $(OBJS)
example: example.o z.lib
$(CC) $(CFLAGS) LINK TO $@ example.o $(LDFLAGS)
minigzip: minigzip.o z.lib
$(CC) $(CFLAGS) LINK TO $@ minigzip.o $(LDFLAGS)
clean:
-delete force quiet *.o example minigzip z.lib foo.gz *.lnk SCOPTIONS
SCOPTIONS: Smakefile
copy to $@ <from <
$(SCOPTIONS)
<
# DO NOT DELETE THIS LINE -- make depend depends on it.
adler32.o: zutil.h zlib.h zconf.h
compress.o: zlib.h zconf.h
crc32.o: zutil.h zlib.h zconf.h
deflate.o: deflate.h zutil.h zlib.h zconf.h
example.o: zlib.h zconf.h
gzio.o: zutil.h zlib.h zconf.h
infblock.o: zutil.h zlib.h zconf.h infblock.h inftrees.h infcodes.h infutil.h
infcodes.o: zutil.h zlib.h zconf.h inftrees.h infutil.h infcodes.h inffast.h
inffast.o: zutil.h zlib.h zconf.h inftrees.h infutil.h inffast.h
inflate.o: zutil.h zlib.h zconf.h infblock.h
inftrees.o: zutil.h zlib.h zconf.h inftrees.h
infutil.o: zutil.h zlib.h zconf.h inftrees.h infutil.h
minigzip.o: zlib.h zconf.h
trees.o: deflate.h zutil.h zlib.h zconf.h
uncompr.o: zlib.h zconf.h
zutil.o: zutil.h zlib.h zconf.h

68
contrib/vmap_extractor_v2/stormlib/zlib/compress.c
View file

@ -1,68 +0,0 @@
/* compress.c -- compress a memory buffer
* Copyright (C) 1995-2002 Jean-loup Gailly.
* For conditions of distribution and use, see copyright notice in zlib.h
*/
/* @(#) $Id$ */
#include "zlib.h"
/* ===========================================================================
Compresses the source buffer into the destination buffer. The level
parameter has the same meaning as in deflateInit. sourceLen is the byte
length of the source buffer. Upon entry, destLen is the total size of the
destination buffer, which must be at least 0.1% larger than sourceLen plus
12 bytes. Upon exit, destLen is the actual size of the compressed buffer.
compress2 returns Z_OK if success, Z_MEM_ERROR if there was not enough
memory, Z_BUF_ERROR if there was not enough room in the output buffer,
Z_STREAM_ERROR if the level parameter is invalid.
*/
int ZEXPORT compress2 (dest, destLen, source, sourceLen, level)
Bytef *dest;
uLongf *destLen;
const Bytef *source;
uLong sourceLen;
int level;
{
z_stream stream;
int err;
stream.next_in = (Bytef*)source;
stream.avail_in = (uInt)sourceLen;
#ifdef MAXSEG_64K
/* Check for source > 64K on 16-bit machine: */
if ((uLong)stream.avail_in != sourceLen) return Z_BUF_ERROR;
#endif
stream.next_out = dest;
stream.avail_out = (uInt)*destLen;
if ((uLong)stream.avail_out != *destLen) return Z_BUF_ERROR;
stream.zalloc = (alloc_func)0;
stream.zfree = (free_func)0;
stream.opaque = (voidpf)0;
err = deflateInit(&stream, level);
if (err != Z_OK) return err;
err = deflate(&stream, Z_FINISH);
if (err != Z_STREAM_END) {
deflateEnd(&stream);
return err == Z_OK ? Z_BUF_ERROR : err;
}
*destLen = stream.total_out;
err = deflateEnd(&stream);
return err;
}
/* ===========================================================================
*/
int ZEXPORT compress (dest, destLen, source, sourceLen)
Bytef *dest;
uLongf *destLen;
const Bytef *source;
uLong sourceLen;
{
return compress2(dest, destLen, source, sourceLen, Z_DEFAULT_COMPRESSION);
}

34
contrib/vmap_extractor_v2/stormlib/zlib/contrib/README.contrib
View file

@ -1,34 +0,0 @@
All files under this contrib directory are UNSUPPORTED. There were
provided by users of zlib and were not tested by the authors of zlib.
Use at your own risk. Please contact the authors of the contributions
for help about these, not the zlib authors. Thanks.
asm386/ by Gilles Vollant <info@winimage.com>
386 asm code replacing longest_match(), for Visual C++ 4.2 and ML 6.11c
asm586/ and asm686/ by Brian Raiter <breadbox@muppetlabs.com>
asm code for Pentium and Pentium Pro
See http://www.muppetlabs.com/~breadbox/software/assembly.html
delphi/ by Bob Dellaca <bobdl@xtra.co.nz>
Support for Delphi
delphi2/ by Davide Moretti <dave@rimini.com>
Another support for C++Builder and Delphi
minizip/ by Gilles Vollant <info@winimage.com>
Mini zip and unzip based on zlib
See http://www.winimage.com/zLibDll/unzip.html
iostream/ by Kevin Ruland <kevin@rodin.wustl.edu>
A C++ I/O streams interface to the zlib gz* functions
iostream2/ by Tyge Løvset <Tyge.Lovset@cmr.no>
Another C++ I/O streams interface
untgz/ by "Pedro A. Aranda Guti\irrez" <paag@tid.es>
A very simple tar.gz file extractor using zlib
visual-basic.txt by Carlos Rios <c_rios@sonda.cl>
How to use compress(), uncompress() and the gz* functions from VB.

559
contrib/vmap_extractor_v2/stormlib/zlib/contrib/asm386/gvmat32.asm
View file

@ -1,559 +0,0 @@
;
; gvmat32.asm -- Asm portion of the optimized longest_match for 32 bits x86
; Copyright (C) 1995-1996 Jean-loup Gailly and Gilles Vollant.
; File written by Gilles Vollant, by modifiying the longest_match
; from Jean-loup Gailly in deflate.c
; It need wmask == 0x7fff
; (assembly code is faster with a fixed wmask)
;
; For Visual C++ 4.2 and ML 6.11c (version in directory \MASM611C of Win95 DDK)
; I compile with : "ml /coff /Zi /c gvmat32.asm"
;
;uInt longest_match_7fff(s, cur_match)
; deflate_state *s;
; IPos cur_match; /* current match */
NbStack equ 76
cur_match equ dword ptr[esp+NbStack-0]
str_s equ dword ptr[esp+NbStack-4]
; 5 dword on top (ret,ebp,esi,edi,ebx)
adrret equ dword ptr[esp+NbStack-8]
pushebp equ dword ptr[esp+NbStack-12]
pushedi equ dword ptr[esp+NbStack-16]
pushesi equ dword ptr[esp+NbStack-20]
pushebx equ dword ptr[esp+NbStack-24]
chain_length equ dword ptr [esp+NbStack-28]
limit equ dword ptr [esp+NbStack-32]
best_len equ dword ptr [esp+NbStack-36]
window equ dword ptr [esp+NbStack-40]
prev equ dword ptr [esp+NbStack-44]
scan_start equ word ptr [esp+NbStack-48]
wmask equ dword ptr [esp+NbStack-52]
match_start_ptr equ dword ptr [esp+NbStack-56]
nice_match equ dword ptr [esp+NbStack-60]
scan equ dword ptr [esp+NbStack-64]
windowlen equ dword ptr [esp+NbStack-68]
match_start equ dword ptr [esp+NbStack-72]
strend equ dword ptr [esp+NbStack-76]
NbStackAdd equ (NbStack-24)
.386p
name gvmatch
.MODEL FLAT
; all the +4 offsets are due to the addition of pending_buf_size (in zlib
; in the deflate_state structure since the asm code was first written
; (if you compile with zlib 1.0.4 or older, remove the +4).
; Note : these value are good with a 8 bytes boundary pack structure
dep_chain_length equ 70h+4
dep_window equ 2ch+4
dep_strstart equ 60h+4
dep_prev_length equ 6ch+4
dep_nice_match equ 84h+4
dep_w_size equ 20h+4
dep_prev equ 34h+4
dep_w_mask equ 28h+4
dep_good_match equ 80h+4
dep_match_start equ 64h+4
dep_lookahead equ 68h+4
_TEXT segment
IFDEF NOUNDERLINE
public longest_match_7fff
; public match_init
ELSE
public _longest_match_7fff
; public _match_init
ENDIF
MAX_MATCH equ 258
MIN_MATCH equ 3
MIN_LOOKAHEAD equ (MAX_MATCH+MIN_MATCH+1)
IFDEF NOUNDERLINE
;match_init proc near
; ret
;match_init endp
ELSE
;_match_init proc near
; ret
;_match_init endp
ENDIF
IFDEF NOUNDERLINE
longest_match_7fff proc near
ELSE
_longest_match_7fff proc near
ENDIF
mov edx,[esp+4]
push ebp
push edi
push esi
push ebx
sub esp,NbStackAdd
; initialize or check the variables used in match.asm.
mov ebp,edx
; chain_length = s->max_chain_length
; if (prev_length>=good_match) chain_length >>= 2
mov edx,[ebp+dep_chain_length]
mov ebx,[ebp+dep_prev_length]
cmp [ebp+dep_good_match],ebx
ja noshr
shr edx,2
noshr:
; we increment chain_length because in the asm, the --chain_lenght is in the beginning of the loop
inc edx
mov edi,[ebp+dep_nice_match]
mov chain_length,edx
mov eax,[ebp+dep_lookahead]
cmp eax,edi
; if ((uInt)nice_match > s->lookahead) nice_match = s->lookahead;
jae nolookaheadnicematch
mov edi,eax
nolookaheadnicematch:
; best_len = s->prev_length
mov best_len,ebx
; window = s->window
mov esi,[ebp+dep_window]
mov ecx,[ebp+dep_strstart]
mov window,esi
mov nice_match,edi
; scan = window + strstart
add esi,ecx
mov scan,esi
; dx = *window
mov dx,word ptr [esi]
; bx = *(window+best_len-1)
mov bx,word ptr [esi+ebx-1]
add esi,MAX_MATCH-1
; scan_start = *scan
mov scan_start,dx
; strend = scan + MAX_MATCH-1
mov strend,esi
; bx = scan_end = *(window+best_len-1)
; IPos limit = s->strstart > (IPos)MAX_DIST(s) ?
; s->strstart - (IPos)MAX_DIST(s) : NIL;
mov esi,[ebp+dep_w_size]
sub esi,MIN_LOOKAHEAD
; here esi = MAX_DIST(s)
sub ecx,esi
ja nodist
xor ecx,ecx
nodist:
mov limit,ecx
; prev = s->prev
mov edx,[ebp+dep_prev]
mov prev,edx
;
mov edx,dword ptr [ebp+dep_match_start]
mov bp,scan_start
mov eax,cur_match
mov match_start,edx
mov edx,window
mov edi,edx
add edi,best_len
mov esi,prev
dec edi
; windowlen = window + best_len -1
mov windowlen,edi
jmp beginloop2
align 4
; here, in the loop
; eax = ax = cur_match
; ecx = limit
; bx = scan_end
; bp = scan_start
; edi = windowlen (window + best_len -1)
; esi = prev
;// here; chain_length <=16
normalbeg0add16:
add chain_length,16
jz exitloop
normalbeg0:
cmp word ptr[edi+eax],bx
je normalbeg2noroll
rcontlabnoroll:
; cur_match = prev[cur_match & wmask]
and eax,7fffh
mov ax,word ptr[esi+eax*2]
; if cur_match > limit, go to exitloop
cmp ecx,eax
jnb exitloop
; if --chain_length != 0, go to exitloop
dec chain_length
jnz normalbeg0
jmp exitloop
normalbeg2noroll:
; if (scan_start==*(cur_match+window)) goto normalbeg2
cmp bp,word ptr[edx+eax]
jne rcontlabnoroll
jmp normalbeg2
contloop3:
mov edi,windowlen
; cur_match = prev[cur_match & wmask]
and eax,7fffh
mov ax,word ptr[esi+eax*2]
; if cur_match > limit, go to exitloop
cmp ecx,eax
jnbexitloopshort1:
jnb exitloop
; if --chain_length != 0, go to exitloop
; begin the main loop
beginloop2:
sub chain_length,16+1
; if chain_length <=16, don't use the unrolled loop
jna normalbeg0add16
do16:
cmp word ptr[edi+eax],bx
je normalbeg2dc0
maccn MACRO lab
and eax,7fffh
mov ax,word ptr[esi+eax*2]
cmp ecx,eax
jnb exitloop
cmp word ptr[edi+eax],bx
je lab
ENDM
rcontloop0:
maccn normalbeg2dc1
rcontloop1:
maccn normalbeg2dc2
rcontloop2:
maccn normalbeg2dc3
rcontloop3:
maccn normalbeg2dc4
rcontloop4:
maccn normalbeg2dc5
rcontloop5:
maccn normalbeg2dc6
rcontloop6:
maccn normalbeg2dc7
rcontloop7:
maccn normalbeg2dc8
rcontloop8:
maccn normalbeg2dc9
rcontloop9:
maccn normalbeg2dc10
rcontloop10:
maccn short normalbeg2dc11
rcontloop11:
maccn short normalbeg2dc12
rcontloop12:
maccn short normalbeg2dc13
rcontloop13:
maccn short normalbeg2dc14
rcontloop14:
maccn short normalbeg2dc15
rcontloop15:
and eax,7fffh
mov ax,word ptr[esi+eax*2]
cmp ecx,eax
jnb exitloop
sub chain_length,16
ja do16
jmp normalbeg0add16
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
normbeg MACRO rcontlab,valsub
; if we are here, we know that *(match+best_len-1) == scan_end
cmp bp,word ptr[edx+eax]
; if (match != scan_start) goto rcontlab
jne rcontlab
; calculate the good chain_length, and we'll compare scan and match string
add chain_length,16-valsub
jmp iseq
ENDM
normalbeg2dc11:
normbeg rcontloop11,11
normalbeg2dc12:
normbeg short rcontloop12,12
normalbeg2dc13:
normbeg short rcontloop13,13
normalbeg2dc14:
normbeg short rcontloop14,14
normalbeg2dc15:
normbeg short rcontloop15,15
normalbeg2dc10:
normbeg rcontloop10,10
normalbeg2dc9:
normbeg rcontloop9,9
normalbeg2dc8:
normbeg rcontloop8,8
normalbeg2dc7:
normbeg rcontloop7,7
normalbeg2dc6:
normbeg rcontloop6,6
normalbeg2dc5:
normbeg rcontloop5,5
normalbeg2dc4:
normbeg rcontloop4,4
normalbeg2dc3:
normbeg rcontloop3,3
normalbeg2dc2:
normbeg rcontloop2,2
normalbeg2dc1:
normbeg rcontloop1,1
normalbeg2dc0:
normbeg rcontloop0,0
; we go in normalbeg2 because *(ushf*)(match+best_len-1) == scan_end
normalbeg2:
mov edi,window
cmp bp,word ptr[edi+eax]
jne contloop3 ; if *(ushf*)match != scan_start, continue
iseq:
; if we are here, we know that *(match+best_len-1) == scan_end
; and (match == scan_start)
mov edi,edx
mov esi,scan ; esi = scan
add edi,eax ; edi = window + cur_match = match
mov edx,[esi+3] ; compare manually dword at match+3
xor edx,[edi+3] ; and scan +3
jz begincompare ; if equal, go to long compare
; we will determine the unmatch byte and calculate len (in esi)
or dl,dl
je eq1rr
mov esi,3
jmp trfinval
eq1rr:
or dx,dx
je eq1
mov esi,4
jmp trfinval
eq1:
and edx,0ffffffh
jz eq11
mov esi,5
jmp trfinval
eq11:
mov esi,6
jmp trfinval
begincompare:
; here we now scan and match begin same
add edi,6
add esi,6
mov ecx,(MAX_MATCH-(2+4))/4 ; scan for at most MAX_MATCH bytes
repe cmpsd ; loop until mismatch
je trfin ; go to trfin if not unmatch
; we determine the unmatch byte
sub esi,4
mov edx,[edi-4]
xor edx,[esi]
or dl,dl
jnz trfin
inc esi
or dx,dx
jnz trfin
inc esi
and edx,0ffffffh
jnz trfin
inc esi
trfin:
sub esi,scan ; esi = len
trfinval:
; here we have finised compare, and esi contain len of equal string
cmp esi,best_len ; if len > best_len, go newbestlen
ja short newbestlen
; now we restore edx, ecx and esi, for the big loop
mov esi,prev
mov ecx,limit
mov edx,window
jmp contloop3
newbestlen:
mov best_len,esi ; len become best_len
mov match_start,eax ; save new position as match_start
cmp esi,nice_match ; if best_len >= nice_match, exit
jae exitloop
mov ecx,scan
mov edx,window ; restore edx=window
add ecx,esi
add esi,edx
dec esi
mov windowlen,esi ; windowlen = window + best_len-1
mov bx,[ecx-1] ; bx = *(scan+best_len-1) = scan_end
; now we restore ecx and esi, for the big loop :
mov esi,prev
mov ecx,limit
jmp contloop3
exitloop:
; exit : s->match_start=match_start
mov ebx,match_start
mov ebp,str_s
mov ecx,best_len
mov dword ptr [ebp+dep_match_start],ebx
mov eax,dword ptr [ebp+dep_lookahead]
cmp ecx,eax
ja minexlo
mov eax,ecx
minexlo:
; return min(best_len,s->lookahead)
; restore stack and register ebx,esi,edi,ebp
add esp,NbStackAdd
pop ebx
pop esi
pop edi
pop ebp
ret
InfoAuthor:
; please don't remove this string !
; Your are free use gvmat32 in any fre or commercial apps if you don't remove the string in the binary!
db 0dh,0ah,"GVMat32 optimised assembly code written 1996-98 by Gilles Vollant",0dh,0ah
IFDEF NOUNDERLINE
longest_match_7fff endp
ELSE
_longest_match_7fff endp
ENDIF
IFDEF NOUNDERLINE
cpudetect32 proc near
ELSE
_cpudetect32 proc near
ENDIF
pushfd ; push original EFLAGS
pop eax ; get original EFLAGS
mov ecx, eax ; save original EFLAGS
xor eax, 40000h ; flip AC bit in EFLAGS
push eax ; save new EFLAGS value on stack
popfd ; replace current EFLAGS value
pushfd ; get new EFLAGS
pop eax ; store new EFLAGS in EAX
xor eax, ecx ; can’t toggle AC bit, processor=80386
jz end_cpu_is_386 ; jump if 80386 processor
push ecx
popfd ; restore AC bit in EFLAGS first
pushfd
pushfd
pop ecx
mov eax, ecx ; get original EFLAGS
xor eax, 200000h ; flip ID bit in EFLAGS
push eax ; save new EFLAGS value on stack
popfd ; replace current EFLAGS value
pushfd ; get new EFLAGS
pop eax ; store new EFLAGS in EAX
popfd ; restore original EFLAGS
xor eax, ecx ; can’t toggle ID bit,
je is_old_486 ; processor=old
mov eax,1
db 0fh,0a2h ;CPUID
exitcpudetect:
ret
end_cpu_is_386:
mov eax,0300h
jmp exitcpudetect
is_old_486:
mov eax,0400h
jmp exitcpudetect
IFDEF NOUNDERLINE
cpudetect32 endp
ELSE
_cpudetect32 endp
ENDIF
_TEXT ends
end

200
contrib/vmap_extractor_v2/stormlib/zlib/contrib/asm386/gvmat32c.c
View file

@ -1,200 +0,0 @@
/* gvmat32.c -- C portion of the optimized longest_match for 32 bits x86
* Copyright (C) 1995-1996 Jean-loup Gailly and Gilles Vollant.
* File written by Gilles Vollant, by modifiying the longest_match
* from Jean-loup Gailly in deflate.c
* it prepare all parameters and call the assembly longest_match_gvasm
* longest_match execute standard C code is wmask != 0x7fff
* (assembly code is faster with a fixed wmask)
*
*/
#include "deflate.h"
#undef FAR
#include <windows.h>
#ifdef ASMV
#define NIL 0
#define UNALIGNED_OK
/* if your C compiler don't add underline before function name,
define ADD_UNDERLINE_ASMFUNC */
#ifdef ADD_UNDERLINE_ASMFUNC
#define longest_match_7fff _longest_match_7fff
#endif
void match_init()
{
}
unsigned long cpudetect32();
uInt longest_match_c(
deflate_state *s,
IPos cur_match); /* current match */
uInt longest_match_7fff(
deflate_state *s,
IPos cur_match); /* current match */
uInt longest_match(
deflate_state *s,
IPos cur_match) /* current match */
{
static uInt iIsPPro=2;
if ((s->w_mask == 0x7fff) && (iIsPPro==0))
return longest_match_7fff(s,cur_match);
if (iIsPPro==2)
iIsPPro = (((cpudetect32()/0x100)&0xf)>=6) ? 1 : 0;
return longest_match_c(s,cur_match);
}
uInt longest_match_c(s, cur_match)
deflate_state *s;
IPos cur_match; /* current match */
{
unsigned chain_length = s->max_chain_length;/* max hash chain length */
register Bytef *scan = s->window + s->strstart; /* current string */
register Bytef *match; /* matched string */
register int len; /* length of current match */
int best_len = s->prev_length; /* best match length so far */
int nice_match = s->nice_match; /* stop if match long enough */
IPos limit = s->strstart > (IPos)MAX_DIST(s) ?
s->strstart - (IPos)MAX_DIST(s) : NIL;
/* Stop when cur_match becomes <= limit. To simplify the code,
* we prevent matches with the string of window index 0.
*/
Posf *prev = s->prev;
uInt wmask = s->w_mask;
#ifdef UNALIGNED_OK
/* Compare two bytes at a time. Note: this is not always beneficial.
* Try with and without -DUNALIGNED_OK to check.
*/
register Bytef *strend = s->window + s->strstart + MAX_MATCH - 1;
register ush scan_start = *(ushf*)scan;
register ush scan_end = *(ushf*)(scan+best_len-1);
#else
register Bytef *strend = s->window + s->strstart + MAX_MATCH;
register Byte scan_end1 = scan[best_len-1];
register Byte scan_end = scan[best_len];
#endif
/* The code is optimized for HASH_BITS >= 8 and MAX_MATCH-2 multiple of 16.
* It is easy to get rid of this optimization if necessary.
*/
Assert(s->hash_bits >= 8 && MAX_MATCH == 258, "Code too clever");
/* Do not waste too much time if we already have a good match: */
if (s->prev_length >= s->good_match) {
chain_length >>= 2;
}
/* Do not look for matches beyond the end of the input. This is necessary
* to make deflate deterministic.
*/
if ((uInt)nice_match > s->lookahead) nice_match = s->lookahead;
Assert((ulg)s->strstart <= s->window_size-MIN_LOOKAHEAD, "need lookahead");
do {
Assert(cur_match < s->strstart, "no future");
match = s->window + cur_match;
/* Skip to next match if the match length cannot increase
* or if the match length is less than 2:
*/
#if (defined(UNALIGNED_OK) && MAX_MATCH == 258)
/* This code assumes sizeof(unsigned short) == 2. Do not use
* UNALIGNED_OK if your compiler uses a different size.
*/
if (*(ushf*)(match+best_len-1) != scan_end ||
*(ushf*)match != scan_start) continue;
/* It is not necessary to compare scan[2] and match[2] since they are
* always equal when the other bytes match, given that the hash keys
* are equal and that HASH_BITS >= 8. Compare 2 bytes at a time at
* strstart+3, +5, ... up to strstart+257. We check for insufficient
* lookahead only every 4th comparison; the 128th check will be made
* at strstart+257. If MAX_MATCH-2 is not a multiple of 8, it is
* necessary to put more guard bytes at the end of the window, or
* to check more often for insufficient lookahead.
*/
Assert(scan[2] == match[2], "scan[2]?");
scan++, match++;
do {
} while (*(ushf*)(scan+=2) == *(ushf*)(match+=2) &&
*(ushf*)(scan+=2) == *(ushf*)(match+=2) &&
*(ushf*)(scan+=2) == *(ushf*)(match+=2) &&
*(ushf*)(scan+=2) == *(ushf*)(match+=2) &&
scan < strend);
/* The funny "do {}" generates better code on most compilers */
/* Here, scan <= window+strstart+257 */
Assert(scan <= s->window+(unsigned)(s->window_size-1), "wild scan");
if (*scan == *match) scan++;
len = (MAX_MATCH - 1) - (int)(strend-scan);
scan = strend - (MAX_MATCH-1);
#else /* UNALIGNED_OK */
if (match[best_len] != scan_end ||
match[best_len-1] != scan_end1 ||
*match != *scan ||
*++match != scan[1]) continue;
/* The check at best_len-1 can be removed because it will be made
* again later. (This heuristic is not always a win.)
* It is not necessary to compare scan[2] and match[2] since they
* are always equal when the other bytes match, given that
* the hash keys are equal and that HASH_BITS >= 8.
*/
scan += 2, match++;
Assert(*scan == *match, "match[2]?");
/* We check for insufficient lookahead only every 8th comparison;
* the 256th check will be made at strstart+258.
*/
do {
} while (*++scan == *++match && *++scan == *++match &&
*++scan == *++match && *++scan == *++match &&
*++scan == *++match && *++scan == *++match &&
*++scan == *++match && *++scan == *++match &&
scan < strend);
Assert(scan <= s->window+(unsigned)(s->window_size-1), "wild scan");
len = MAX_MATCH - (int)(strend - scan);
scan = strend - MAX_MATCH;
#endif /* UNALIGNED_OK */
if (len > best_len) {
s->match_start = cur_match;
best_len = len;
if (len >= nice_match) break;
#ifdef UNALIGNED_OK
scan_end = *(ushf*)(scan+best_len-1);
#else
scan_end1 = scan[best_len-1];
scan_end = scan[best_len];
#endif
}
} while ((cur_match = prev[cur_match & wmask]) > limit
&& --chain_length != 0);
if ((uInt)best_len <= s->lookahead) return (uInt)best_len;
return s->lookahead;
}
#endif /* ASMV */

Some files were not shown because too many files have changed in this diff Show more