1. Introduction *)
2. Data Structures 
   2.1 Files
   2.2 Basic Types And Conventions
   2.3 String Type
   2.4 Offset Type
   2.5 SchemaMetafile_Harddrive.csc File Format
         2.5.1 File Format
         2.5.2 Sound Tags
         2.5.3 Tagged Sounds
         2.5.4 Mission Sounds
         2.5.5 Sound Information
   2.6 SchemaMetafile.csc File Format *)
3. Extracting/Inserting Sounds

Appendix A. Microsoft WAV File Format
Appendix B. DVI4/IMA ADPCM Audio Format
   B.1 The Deus Ex 2 Audio Format
   B.2 Decoding DVI4/IMA ADPCM
   B.3 Encoding DVI4/IMA ADPCM *)
Appendix C. Acknowledgements
*) Work in progress. Expect this to appear/improve in later revisions.

Deus Ex 2 uses a mixture of different file types for audio. Two basic file types are used: Ogg Vorbis (.OGG files) and Wave (.WAV files).

Ogg Vorbis is an open source audio compression standard (actually: Ogg = the container format, Vorbis = the compression) which gives better sound quality at equal bitrate when compared to mp3. The format is getting increasingly popular by both consumers and (game) developers because of the quality and the free nature of the standard (no fees!). A lot of DX2 sounds are encoded in Ogg Vorbis. SDKs for encoding/decoding Ogg Vorbis files are freely available at the website of the developers; please see http://www.xiph.org/ for more information.

The wave format, or rather RIFF wave format is a standard developed by Microsoft/IBM. It supports not only the bog-standard PCM audio (uncompressed), but also several non-PCM audio (compressed) types. Microsoft Windows already has codecs for a number of these standards built in. Developers are allowed to use their own compression standard (and are in fact provided with the means to do so using the 'audio compression manager').

Guess what? Deus Ex 2 seems to do just that. Besides some normal PCM wave files, it also uses wave file with audio format 0x0069, which is not one of the standard codecs provided by Microsoft. The format used is either the now obsolete 'VoxWare Byte Aligned' standard or Ionstorms custom audio codec. Considering the obsolete status of the first, a custom format is expected (note: specification of the VoxWare format were not found on the net, so it cannot be verified at this time).

What can be found out about this custom format?
- nBitsPerSample is always set to 4. This already indicates some sort of ADPCM compression method, since the existing methods (Microsoft and IMA) always compress from 16 bits to 4 (or 3) bits.
- nAvgBytesPerSec is always the same, depends only on samplerate, this suggests a fixed compression rate. Again: possibly ADPCM.
- wExtraParamsSize is always 2, extra parameters are always 0x0040 (64). This fits with IMA ADPCM specs which say the extra parameter is a WORD with the number of samples per block per channel. However, the value does NOT fit, the specs say they are supposed to fit the rule 'Samples%8==1' and they don't.

The primary suspect for the custom audio codec is now IMA ADPCM with some (slight) adjustments. Now, take a look inside the file DX2.exe version 1.0 at offset 0x005525F0. Guess what that is...this is the 'stepsizeTable' followed by the 'indexTable' exactly as used in the IMA ADPCM specifications. This confirms this audio codec is used.

There is a variant of IMA ADCPM called DVI4 or DVI ADPCM. It is basically the same but has one significant difference. IMA ADPCM uses this header in a block of audio data:

   struct SIMABlockHeader
   {
      int   nSample;          // first sample of the block.
      short nStep;            // index into 'stepsize table'.
      short nReserved;        // reserved, assumed 0.
   };

Followed by groups of 8 samples (4 bytes) until the end of the block. As a consequence, an IMA block always contains a multiple of 8 samples per block PLUS ONE (the extra sample in the block header). DVI4 does not use this value in the header as an actual sample, but as a prediction. The block header becomes:

   struct SDVI4BlockHeader
   {
      int   nPredict;         // prediction used to calculate first sample .
      short nStep;            // index into 'stepsize table'.
      short nReserved;        // reserved, 0 when writing, ignore when reading.
   };

Again followed by groups of 8 samples (4 bytes) until the end of the block. So here are always a multiple of 8 samples per block which fits nicely with the value provided by the extra parameter 0x0040.

  0            4           8              12             16             20             24
  [ch0 header ][ch1 header][ch0 8 samples][ch1 8 samples][ch0 8 samples][ch1 8 samples]...

This (de-)compression method gives good results on the sounds of Deus Ex 2. All 0x0069 sounds can be decoded this way and they seem to be correct for the most part.

Big problem: the results for STEREO 0x0069 sounds are NOT satisfactory. Although the method appears to be correct, the resulting sounds exhibit extreme clipping (too often). Since there are only a few stereo 0x0069 sounds in the Deus Ex 2 files, first thing to research is whether these sounds are actually used by the game or not. Expect an update on this in the coming weeks.

   // quantizer lookup table.
   const int stepsizeTable[89] =
   {
      7     ,  8     ,  9     ,  10    ,  11    ,  12    ,  13    ,  14    ,  16    ,
      17    ,  19    ,  21    ,  23    ,  25    ,  28    ,  31    ,  34    ,  37    ,
      41    ,  45    ,  50    ,  55    ,  60    ,  66    ,  73    ,  80    ,  88    ,
      97    ,  107   ,  118   ,  130   ,  143   ,  157   ,  173   ,  190   ,  209   ,
      230   ,  253   ,  279   ,  307   ,  337   ,  371   ,  408   ,  449   ,  494   ,
      544   ,  598   ,  658   ,  724   ,  796   ,  876   ,  963   ,  1060  ,  1166  ,
      1282  ,  1411  ,  1552  ,  1707  ,  1878  ,  2066  ,  2272  ,  2499  ,  2749  ,
      3024  ,  3327  ,  3660  ,  4026  ,  4428  ,  4871  ,  5358  ,  5894  ,  6484  ,
      7132  ,  7845  ,  8630  ,  9493  ,  10442 ,  11487 ,  12635 ,  13899 ,  15289 ,
      16818 ,  18500 ,  20350 ,  22385 ,  24623 ,  27086 ,  29794 ,  32767
   };

   // table of index changes.
   const int indexTable[16] =
   {
      -1, -1, -1, -1, +2, +4, +6, +8,
      -1, -1, -1, -1, +2, +4, +6, +8
   };

   header            // block header, see SIMABlockHeader or SDVI4BlockHeader structs.
   int newSample     // sample for output.
   short index       // index into stepsizeTable.
   int stepsize      // stepsize value.
   NIBBLE sample     // current sample (NOTE: two samples of 4 bits are packed into a BYTE of
                     // the data block, process the low nibble first, then the high nibble).
   int delta         // difference between consecutive (output) samples.

   // initial state.
   get block header -> header
   if(method==IMA)
   {
      newSample = header.nSample 
      output newSample
   }
   else if(method==DVI4)
      newSample = header.nPredict
   index = header.nStep
   stepsize = stepsizeTable[index]

   // process the samples.
   for every sample in the block
   {
      // get sample.
      get sample -> sample

      // calculate delta value.
      delta = 0
      if(sample & 4)    delta += stepsize
      if(sample & 2)    delta += stepsize>>1
      if(sample & 1)    delta += stepsize>>2
      delta += stepsize>>3;
      if(sample & 8)    delta = -delta  

      // calculate and clip output sample.
      newSample += delta
      if( newSample > 32767 )          newSample = 32767
      else if( newSample < -32768 )    newSample = -32768
      // NOTE: watch those 16 bit limits here -> better use temporary long to add the delta...

      // output the new sample.
      output new Sample

      // new stepsize.
      index += indexTable[sample]
      if( index < 0 )         index = 0
      else if( index > 88 )   index = 88
      stepsize = stepsizeTable[index]
   }