 | Edinburgh Speech Tools Library |  | ||
|---|---|---|---|---|
| Prev | Next | |||
EST offers two classes for handling and storing speech information of all types: Waveforms and Tracks. Both are basically matrices with one dimension representing time, the other representing a particular channel and the value at that position representing an amplitude. There are signficant differences between them, however, that makes the use of two separate classes preferable to one.
Waveforms store digitalled sampled acoustic waveforms. They are composed of a matrix of shorts, where rows represent individual samples and columns represent channels. Waves can have arbitrarily many channels, though 1 (mono) and 2 (stereo) are the most common. Waves are stored as shorts as this is the most common file format, which ensures fast compatibility with most file formats and hardware. As each sample is representing by a 16-bit short, the dynamic range of a wave is 96dB.
The track class is used to represent the outcome of a signal processing operation on a section of speech. It can be thought of as representing a series of frames, where each frame represents signal processing information at a specified time point.
Each frame is a set of ordered coefficients, which represent the output of a signal processing operation on a single section of speech. For example, a frame may represent a spectrum, a cepstrum, or a set of linear predication coefficients. An alternative view is to visualise the track as a set of channels, where each channel represents a how a particular type of information varies with time. For instance, a channel might represent how the energy between 500Hz and 600Hz varies over the course of an utterance.
Frames and Channels are stored as a matrix of floats, where each point in the matrix represents the amplitude of a given frame and a given channel.
In addition to the amplitude matrix, tracks also contain a timearray, which has the same number of elements as frames in the amplitude matrix. The time array is aligned one-to-one with the frames. Each position in the time array represents the time of its frame. In many forms of signal processing, frames are at fixed intervals (often 10ms), and in such cases it would be possible to store this as a single global value. However, the track class is extremely general in terms of time positions and allows frames to be spaced irregularly, which is particularly useful when dealing with pitch-synchronous processing.
The track class also contains a break/value array, each element of which also as a one-to-one correspondence with a frame. In many representations some frames have undefined values, and the break/value array is used to represent this. For example, F0 contours and formants do not have values during unvoiced sections of speech, and hence frames representing unvoiced sections may be tagged as breaks in the break/value array. By default, it is assumed that all amplitudes are defined and hence no breaks are set at contour initialisation or resizing.
In time, this will be replaced by the more general Auxiliary Matrix.
trackmaps etc.
It is inappropriate to store certain information in the amplitude array.
The track class provides an easy mechanism for dealing with a single portion of the track at a time. If we have say a track with 10 channels and 500 frames, it is possible to assign a vector to any single frame or channel, or to assign a sub-track to any contiguous set of frames or channels. Any values that are changed in the frame or channel vectors or sub-track, will affect the underlying tack. It is of course possible to copy values in and out if values need to be changed without changing the underlying track.