Latest posts of: Platinum Lucario

Quote from: libertyernie on March 06, 2017, 07:01:04 AM

Do you have a link to some sort of specification for the header formats?

I assume vgmstream doesn't have this problem, right?

I'm pretty sure vgmstream doesn't have this issue. And neither does Uwizard.

As for specifications, I can't actually find any, but the closest I can get to, in regards to this format is here: https://gitlab.kode54.net/kode54/vgmstream/blob/master/src/meta/bcstm.c and here: https://gitlab.kode54.net/kode54/vgmstream/blob/master/src/meta/bfstm.c

I would assume there might be some documentation in the NW4C and NW4F official SoundMaker documents. I'll check them out and post it here when I've found it.

Edit: Found it!

Cafe DSP-ADPCM Stream (FSTM):

Quote

3 Data formats
3.1 WAV files DSPADPCM converts standard WAV files into DSP-ADPCM format. The WAV files must contain mono, 16-bit PCM data.

3.2 AIFF files DSPADPCM can also convert AIFF files into DSP-ADPCM format. The AIFF files must contain mono, 16-bit PCM data. Note: Loop points in the AIFF file will be read automatically and programmed into the header of the DSP-ADPCM output file.

3.3 DSP-ADPCM files When converting data into DSP-ADPCM format, the tool will preface the output data with a header. The structure of the header is defined below:

Code 1 - DSPADPCM header file

Code:

// all data in this structure is in BIG ENDIAN FORMAT!!!! 
 
typedef struct 
{ 
// for header generation during decode
     u32 num_samples;       // total number of RAW samples
     u32 num_adpcm_nibbles; // number of ADPCM nibbles (including frame headers)
     u32 sample_rate;       // Sample rate, in Hz 
 
// DSP addressing and decode context
     u16 loop_flag;    // 1=LOOPED, 0=NOT LOOPED
     u16 format;       // Always 0x0000, for ADPCM
     u32 sa;           // Start offset address for looped samples (zero for non-looped)
     u32 ea;           // End offset address for looped samples
     u32 ca;           // always zero
     u16 coef[16];     // decode coefficients (eight pairs of 16-bit words) 
 
// DSP decoder initial state
     u16 gain;         // always zero for ADPCM
     u16 ps;           // predictor/scale
     u16 yn1;          // sample history
     u16 yn2;          // sample history 
 
// DSP decoder loop context
     u16 lps;          // predictor/scale for loop context
     u16 lyn1;         // sample history (n-1) for loop context
     u16 lyn2;         // sample history (n-2) for loop context 
 
     u16 pad[11];      // reserved 
 
} DSPADPCM; 
 
// Header is 96 bytes long

This header contains information needed by the Nintendo Cafe audio DSP to decode and play the associated sample.
Note: All data in the header is stored in big-endian format. This facilitates transfer of the data to a Nintendo Cafe runtime application. Much of the data may be used verbatim to program the DSP for sample playback. Consult the Audio Library (AX) section in this guide for application details. When decoding DSP-ADPCM data into WAV or AIFF format, the tool will assume that this header is present at the start of the DSP-ADPCM file. The DSPADPCM tool needs the first two parameters of the header to regenerate WAV header information during decode:

num_samples Number of raw, uncompressed samples in the file. Used for WAV/AIFF header generation during decode.

num_adpcm_nibbles

Number of ADPCM nibbles (including frame headers) generated for this sample.

Note:

You must round this up to the next multiple of 8 bytes to get the actual length of the data in the file because DSPADPCM only generates complete frames.

sampling_rate Sampling rate of the data, expressed in Hertz. Used for WAV/AIFF header generation during decode.

The remaining parameters are required by the audio DSP to decode and play the associated ADPCM sample data:

loop_flag Specifies whether or not the sample is looped. This parameter is stored in big-endian format and is used by the DSP for sample playback.

format Specifies the data format of the sample. Always zero. Used by the DSP for sample playback.

sa Loop start offset, in nibbles. The user application must add the main memory address of the sample data before the DSP can use it.

ea Loop end offset, in nibbles. The user application must add the main memory address of the sample data before the DSP can use it.

ca Initial offset value, in nibbles. Always zero. The user application must add the main memory address of the sample data before the DSP can use it.

coef[16]

Decoder coefficients. This coefficient corresponds to AXPBADPCM Structure member a[][] in the following way.

a[0][0] = coef[0];

a[0][1] = coef[1];

a[1][0] = coef[2];

a[1][1] = coef[3];

a[2][0] = coef[4];

a[2][1] = coef[5];

a[3][0] = coef[6];

a[3][1] = coef[7];

a[4][0] = coef[8];

a[4][1] = coef[9];

a[5][0] = coef[10];

a[5][1] = coef[11];

a[6][0] = coef[12];

a[6][1] = coef[13];

a[7][0] = coef[14];

a[7][1] = coef[15];

gain Gain factor. Always zero for ADPCM samples. ps Predictor and scale. This will be initialized to the predictor and scale value of the sample’s first frame.

yn1 History data; used to maintain decoder state during sample playback.

yn2 History data; used to maintain decoder state during sample playback.

lps Predictor/scale for the loop point frame. If the sample does not loop, this value is zero.

lyn1 History data for the loop point. If the sample does not loop, this value is zero.

lyn2 History data for the loop point. If the sample does not loop, this value is zero.

Some notes about decoder addressing:

When processing ADPCM samples, the DSP will address memory as 4-bit nibbles.
The values for sa, ea, and ca generated by DSPADPCM are nibble-offsets which already account for the extra space needed for ADPCM frame headers. For example, the one hundredth sample does not
refer to the one hundredth nibble in the sample data; the one hundredth sample would actually be the one hundred-sixteenth nibble.
The sa, ea, and ca values are offsets. When encoding data, DSPADPCM cannot know where the sample will be placed in memory. The user application is therefore responsible for adding a main memory address (in nibbles) to these offsets before the DSP can access the sample.
Note that individual ADPCM sound effects must start on 8-byte boundaries and must be at least a multiple of 8 bytes in length. Thus, when loading one or more ADPCM samples into memory, the samples must be packed such that the start of each sample falls on an 8-byte boundary.

CTR DSP-ADPCM Stream (CSTM):

Quote

Overview
WaveCodecCtr.dll is a Win32 run-time dynamic link library (DLL). This library provides an API for encoding and decoding between 16-bit signed (little-endian) sampling data and the DSP ADPCM compressed format.
The DSP ADPCM format is a compressed format that can be played by a CTR system's DSP. It can result in data sizes that are 1/3.5 of 16-bit PCM. This format can be used through the nn::snd::Voice class and the nn::snd::WaveBuffer structure.
This DLL is for developers who want to develop their own tools for creating and previewing DSP ADPCM sampling data. Note that this DLL is single-threaded.
ctr_WaveConverter uses the features of this DLL to create BCWAV files in ADPCM format.
Exported Functions
getBytesForAdpcmBuffer
getBytesForAdpcmSamples
getBytesForPcmBuffer
getBytesForPcmSamples
getSampleForAdpcmNibble
getNibbleAddress
getLoopContext
encode
decode
getBytesForAdpcmBuffer
Syntax

Code:

u32 getBytesForAdpcmBuffer(u32 samples);

Arguments

	Name	Description
in	samples	Number of (16-bit PCM) samples to encode.

Return Values
The number of bytes required to store the DSP ADPCM-encoded samples.
Description
getBytesForAdpcmBuffer calculates and returns the number of bytes required to store the sampling data that are being DSP ADPCM-encoded.
Note that the number of bytes will be a multiple of the 8-byte length of DSP ADPCM frames.
The actual length (in bytes) of the sampling data being encoded will probably be smaller than the value returned by this function. (For more information, see the section getBytesForAdpcmSamples.)
Use this function before encoding to allocate storage for the result.
getBytesForAdpcmSamples
Syntax

Code:

u32 getBytesForAdpcmSamples(u32 samples);

Arguments

Name	Description
in	samples	Number of (16-bit PCM) samples to encode.

Return Values
The actual length (in bytes) of the encoded sampling data.
Description
getBytesForAdpcmSamples returns the actual number of bytes occupied by the sampling data that are being DSP ADPCM-encoded.
getBytesForPcmBuffer
Syntax

Code:

u32 getBytesForPcmBuffer(u32 samples);

Arguments

	Name	Description
in	samples	The number of samples being decoded.

Return Values
The predicted length (in bytes) of the DSP ADPCM sampling data being decoded.
Description
getBytesForPcmBuffer calculates and returns the number of bytes of data that must be stored after the number of samples passed in the argument have been decoded from the DSP ADPCM format.
getBytesForPcmSamples
Syntax

Code:

u32 getBytesForPcmSamples(u32 samples);

Arguments

	Name	Description
in	samples	Number of samples.

Return Values
The length (in bytes) of the value specified in the sampling data.
Description
getBytesForPcmSamples returns the number of bytes for copying the specified user application value in the sampling data.
getSampleForAdpcmNibble
Syntax

Code:

u32 getSampleForAdpcmNibble(u32 nibble);

Arguments

	Name	Description
in	nibble	ADPCM nibble offset, including frame headers.

Return Values
The zero-based sample index for the corresponding ADPCM sampling data.
Description
getSampleForAdpcmNibble returns the number of zero-based samples for the corresponding ADPCM nibble.
getNibbleAddress
Syntax

Code:

u32 getNibbleAddress(u32 samples);

Arguments

	Name	Description
in	samples	16-bit PCM address offset. 0 is the first sample. 100 is the 101st sample.

Return Values
Corresponding nibble address.
Description
getNibbleAddress calculates and returns the nibble corresponding to the sampling data, as based on the offset passed to the argument.
For example, the 100th sample (where the count starts at 0) corresponds to the 116th nibble (where the count also starts at 0). Note that the calculated nibble address incorporates the 2-nibble frame header that is already in DSP ADPCM compressed format.
One more example: The 0th sample corresponds to a nibble offset of 2, which is the 3rd nibble (counting from 0).
getLoopContext
Syntax

Code:

typedef struct 
{ 
    // start context 
    s16 coef[16]; 
    u16 gain; 
    u16 pred_scale; 
    s16 yn1; 
    s16 yn2; 

    // loop context 
    u16 loop_pred_scale; 
    s16 loop_yn1; 
    s16 loop_yn2; 
} ADPCMINFO;  

void getLoopContext( 
    u8          *src,       // location of ADPCM buffer in RAM 
    ADPCMINFO   *cxt,       // location of adpcminfo 
    u32         samples     // samples to desired context 
);

Arguments

	Name	Description
in	src	Pointer to the buffer containing the DSP ADPCM encoded sample.
out	cxt	The pointer to the ADPCMINFO structure. The getLoopContext function places the loop context information inside this structure.
in	samples	The offset from the starting position of the loop in the sampling data. This loop starting position is not a nibble address, but rather the raw sample number where the loop starts (in other words, the first sample played in the loop). If the loop begins at the very first sample, the offset is zero. If the loop begins at the 101st sample, the offset is 100.

Return Values
None.
Description
getLoopContext returns the DSP ADPCM loop context, based on the offset in the sample data given to the argument.
encode
Syntax

Code:

typedef struct 
{ 
    // start context 
    s16 coef[16]; 
    u16 gain; 
    u16 pred_scale; 
    s16 yn1; 
    s16 yn2; 

    // loop context 
    u16 loop_pred_scale; 
    s16 loop_yn1; 
    s16 loop_yn2; 
} ADPCMINFO;  

void encode( 
    s16         *src,       // location of source samples (16bit PCM signed little endian) 
    u8          *dst,       // location of destination buffer 
    ADPCMINFO   *cxt,       // location of adpcm info 
    u32         samples     // number of samples to encode 
);

Arguments

	Name	Description
in	src	Pointer to the buffer for signed little-endian 16-bit PCM sampling data.
out	dst	The pointer to the buffer where the DSP ADPCM encoded data are output. The application must allocate memory for this buffer. You must calculate the buffer size using getBytesForAdpcmBuffer.
in	cxt	Pointer to the ADPCMINFO structure. The encode function stores sampling data coefficients and context information in this structure. Note that a number of parameters (gain, yn1, and yn2) are always 0. Before decoding commences, the corresponding registers in the DSP decoder hardware must be cleared. These parameters are included in the structure to force this to happen. After encode has been called, the loop context parameters always become 0. To set these values, you must call getLoopContext after the sampling data have been looped.
in	samples	Number of samples to encode.

Return Values
None.
Description
encode compresses 16-bit PCM data into DSP ADPCM format.
decode
Syntax

Code:

typedef struct 
{ 
    // start context 
    s16 coef[16]; 
    u16 gain; 
    u16 pred_scale; 
    s16 yn1; 
    s16 yn2; 

    // loop context 
    u16 loop_pred_scale; 
    s16 loop_yn1; 
    s16 loop_yn2; 
} ADPCMINFO;

void decode( 
    u8          *src,   // location of encoded source samples 
    s16         *dst,   // location of destination buffer (16 bits / sample) 
    ADPCMINFO   *cxt,   // location of adpcm info 
    u32         samples // number of samples to decode 
);

Arguments

	Name	Description
in	src	Pointer to the buffer containing the DSP ADPCM data.
out	dst	Pointer to a buffer that has been allocated for storing the uncompressed PCM data. The size of this buffer is calculated by calling getBytesForPcmBuffer.
in	cxt	Pointer to the ADPCMINFO structure. This structure must contain the coefficient and initial state data that corresponds to the sample being decoded.
in	samples	Number of samples to decode.

Return Values
None.
Description
decode is used for decoding DSP ADPCM sample data into signed, little-endian, 16-bit PCM data.
Sample Code
WaveCodecCtr.dll has been created for use with Win32 applications. This section provides some samples.
Initialization
Encoding
Decoding
Initialization
Applications that call DLLs must comply with Win32 rules when calling WaveCodecCtr.dll.

Code:

typedef signed short   s16;
typedef unsigned char  u8;
typedef unsigned short u16;
typedef unsigned long  u32;

/* ---------------------------------------------------------------

  Import the WaveCodecCtr.dll API.
  
 ---------------------------------------------------------------- */
typedef struct
{
    // start context
    s16 coef[16];
    u16 gain;
    u16 pred_scale;
    s16 yn1;
    s16 yn2;

    // loop context
    u16 loop_pred_scale;
    s16 loop_yn1;
    s16 loop_yn2;

} ADPCMINFO;

static HINSTANCE hDll; 
typedef u32 (*lpFunc1)(u32); 
typedef u32 (*lpFunc2)(void); 
typedef void (*lpFunc3)(s16*, u8*, ADPCMINFO*, u32); 
typedef void (*lpFunc4)(u8*, s16*, ADPCMINFO*, u32); 
typedef void (*lpFunc5)(u8*, ADPCMINFO*, u32); 
lpFunc1 getBytesForAdpcmBuffer; 
lpFunc1 getBytesForAdpcmSamples; 
lpFunc1 getBytesForPcmBuffer;
lpFunc1 getBytesForPcmSamples; 
lpFunc1 getSampleForAdpcmNibble; 
lpFunc1 getNibbleAddress; 
lpFunc2 getBytesForAdpcmInfo; 
lpFunc3 encode; 
lpFunc4 decode; 
lpFunc5 getLoopContext; 

/*--------------------------------------------------------------------------*/ 
void clean_up(void) 
{ 
    if (hDll) 
        FreeLibrary(hDll); 
} 

/*--------------------------------------------------------------------------*/ 
int getDll(void) 
{ 
    hDll = LoadLibrary("WaveCodecCtr.dll"); 
    if (hDll) 
    { 
        if (!(getBytesForAdpcmBuffer = 
                (lpFunc1)GetProcAddress( 
                             hDll, 
                             "getBytesForAdpcmBuffer" 
                             ))) return 1; 
        if (!(getBytesForAdpcmSamples = 
                (lpFunc1)GetProcAddress( 
                             hDll, 
                             "getBytesForAdpcmSamples" 
                             ))) return 1; 
        if (!(getBytesForPcmBuffer = 
                (lpFunc1)GetProcAddress( 
                             hDll, 
                             "getBytesForPcmBuffer" 
                             ))) return 1; 
        if (!(getBytesForPcmSamples = 
                (lpFunc1)GetProcAddress( 
                             hDll, 
                             "getBytesForPcmSamples" 
                             ))) return 1; 
        if (!(getNibbleAddress = 
                (lpFunc1)GetProcAddress( 
                             hDll, 
                             "getNibbleAddress" 
                             ))) return 1; 
        if (!(getSampleForAdpcmNibble = 
                (lpFunc1)GetProcAddress( 
                             hDll, 
                             "getSampleForAdpcmNibble" 
                             ))) return 1; 
        if (!(getBytesForAdpcmInfo = 
                (lpFunc2)GetProcAddress( 
                             hDll, 
                             "getBytesForAdpcmInfo" 
                             ))) return 1; 
        if (!(encode =
                (lpFunc3)GetProcAddress(
                            hDll,
                            "encode"
                            ))) return 1;
        if (!(decode =
                (lpFunc4)GetProcAddress(
                            hDll,
                            "decode"
                            ))) return 1;
        if (!(getLoopContext = 
                (lpFunc5)GetProcAddress( 
                             hDll, 
                             "getLoopContext" 
                             ))) return 1; 
        return(0); 
    } 
    return(1); 
}


/*--------------------------------------------------------------------------*/ 
int _tmain(int argc, _TCHAR* argv[])
{
    if (getDll()) 
    {
        clean_up(); 
        exit(1); 
    } 

    // do stuff here 

    clean_up(); 
}

Encoding
The encoding function assumes that the data are 16-bit, little endian, PCM data (as used for Windows .wav files). If you want the application to encode big-endian data, you must flip the endian before encoding.

Code:

//... loaded WaveCodecCtr.dll 

//... put some PCM buffer in memory, reverse the endian if you have to 
u8 *adpcm = (u8*)malloc(getBytesForAdpcmBuffer(samplesToEncode)); 

if (adpcm) 
{ 
    ADPCMINFO adpcminfo; 

    // ok.. lets encode it! 
    encode(source, adpcm, &adpcminfo, samplesToEncode); 

    // get ADPCM loop context if sample is looped 
    if (samplesToLoopStart) 
            getLoopContext(adpcm, &adpcminfo, samplesToLoopStart); 

    // see how many bytes to store the encoded data stream 
    u32 nBytesToStore = getBytesForAdpcmSamples(samplesToEncode); 

    ... store encoded ADPCM data stream to file 

    ... store ADPCM context to file 

    u32 nibbleStartOffset   = getNibbleAddress(0); 
    u32 nibbleLoopStartOffset = getNibbleAddress(samplesToLoopStart); 
    u32 nibbleEndAddress  = getNibbleAddress(samplesToEncode); 

    ... store nibble addressing to file 

    // don't need the ADPCM buffer anymore 
    free(adpcm); 
} 

... continue

Decoding
The decoding result is output as 16-bit, little-endian, PCM data.

Code:

... loaded WaveCodecCtr.dll 

... put some ADPCM buffer and corresponding ADPCMINFO in memory, 
... ADPCM is byte ordered.. not endian sensitive. 

s16 *pcm = (u8*)malloc(getBytesForPcmBuffer(samplesToDecode)); 

if (pcm) 
{ 
    // Decode
    decode(source, pcm, adpcminfo, samplesToDecode); 

    ... store decoded PCM buffer to file 

    // Free the PCM buffer
    free(pcm); 
} 

... continue

Cautions for Encoding and Playing Looped Wave Files
Note the following cautions when encoding looped waveforms and when playing them on the actual hardware.
The information relating to encoding also pertains to the use of the nn::snd::EncodeAdpcmData function for encoding data into DSP ADPCM on the actual hardware.
How to Play Regular Looped Waveforms
Align the Loop Start Position to an 8-byte Boundary
Noise When the Loop Start Equals the Start of the Waveform
Noise When the Loop End Equals the End of the Waveform
How to Play Regular Looped Waveforms
When looping a waveform like the following on the actual hardware:

Code:

    WS      LS                     LE
    |-------|<-------------------->|
  
      WS: Start of waveform
      LS: Loop start position
      LE: Loop end position

[list=1]

WaveBuffer for playing WS to LE
WaveBuffer for playing LS to LE

Prepare two WaveBuffer instances (as shown) and set the order as 1. → 2. in nn::snd::Voice.
[/list]

Code:

    WS                             LE
 1. |------------------------------|    (When the loopFlag of WaveBuffer is false.)
  
            LS                     LE
 2.         |<-------------------->|    (When the loopFlag of WaveBuffer is true.)

Align the Loop Start Position to an 8-byte Boundary
In the DSP ADPCM format, 8 bytes (corresponding to 14 samples) are handled as one set of information. For this reason, the bufferAddress argument of the nn::snd::WaveBuffer function must specify an address with an 8-byte boundary.
On the other hand, when creating WaveBuffer as described in 2, LS is not necessarily at an 8-byte (14-sample) boundary.
If the boundary is not 8 bytes (14 samples), you must displace the LS position to an 8-byte boundary so this restriction is satisfied.
For example, if LS is at the 16th sample, you could slide it to the next 14-sample boundary (28 samples) and top off the waveforms after LE with the moved part before encoding.

Code:

     LS                  LE
     |<----------------->|
      ^^^^
                ↓
         LS'                   LE'
     ----|<------------------->|
                           ^^^^

Noise When the Loop Start Equals the Start of the Waveform
Because of encoder limitations, when the loop start position equals the start of a waveform, you cannot play a clean loop without noise.
If the loop start position equals the start of the waveform, use the technique described in Align the loop start position to an 8-byte boundary to slide LS' to 14, so you can play the loop cleanly without noise.
Noise When the Loop End Equals the End of the Waveform
Because of encoder limitations, when the loop start position equals the end of a waveform, encoding is not stable and noise occurs.
When waveforms continue beyond LE, for the samples argument of the encode function, pass WS to end of waveform rather than WS to LE.

Code:

    WS      LS                     LE   WE
    |-------|<-------------------->|----|
  
      WS:  Start of waveform
      WE : End of waveform
      LS: Loop start position
      LE: Loop end position

    encode( WS address, dst, cxt, WE-WS );

When waveforms do not continue beyond LE, apply the technique introduced in Align the loop start position to an 8-byte boundary to use copied waveforms for about 100 samples from LS, and position them after LE.

Code:

    WS      LS                     LE=WE
    |-------|<-------------------->|
             ^^^^^
                      ↓
    WS      LS                     LE   WE'
    |-------|<-------------------->|----|
                                    ^^^^^

    encode( WS address, dst, cxt, WE'-WS );

I hope some of this info from the official SDK docs and from vgmstream's code will help in how the header and the coding of the DSP-ADPCM binaries are encoded. ^^