summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMonty <xiphmont@xiph.org>2012-02-03 19:49:57 +0000
committerMonty <xiphmont@xiph.org>2012-02-03 19:49:57 +0000
commita9396f251eb5df590709bfa1ad710b0abe530d88 (patch)
tree458f0bba7838de9c93f579ae83ab110944d025cb
parent27af612b9879687bf5e319ef3b8789ee773683ec (diff)
downloadlibvorbis-git-a9396f251eb5df590709bfa1ad710b0abe530d88.tar.gz
Add a clarification to the floor1 decode spec, since I had to re-determine some of what
it says for myself after not thinking about it for 15-ish years. Also, fix an indentation bug that occurred due to TABs sneaking in svn path=/trunk/vorbis/; revision=18181
-rw-r--r--doc/07-floor1.tex12
1 files changed, 11 insertions, 1 deletions
diff --git a/doc/07-floor1.tex b/doc/07-floor1.tex
index fd36314f..76fd4534 100644
--- a/doc/07-floor1.tex
+++ b/doc/07-floor1.tex
@@ -252,6 +252,16 @@ optimizations, implementors are warned to follow the details closely.
Deviation from implementing a strictly equivalent algorithm can result
in serious decoding errors.
+{\em Additional note:} Although predicted values in the prediction
+loop and at the end of step 1 (that is, the values in vector
+\varname{[floor1\_final\_Y]} are inherently limited by the prediction
+algorithm to \[0, \varname{[range]}\), it is possible to abuse the
+setup and codebook machinery to produce negative or over-range
+results. We suggest that decoder implementations guard the values in
+vector \varname{[floor1\_final\_Y]} by clamping each element to \[0,
+\varname{[range]}\) after step 1. Variants of this suggestion are
+acceptable as valid floor1 setups cannot produce out of range values.
+
\begin{description}
\item[step 1: amplitude value synthesis]
@@ -307,7 +317,7 @@ Unwrap the always-positive-or-zero values read from the packet into
\} else [val] is less than [room] \{
- 24) if ([val] is odd) \{
+ 24) if ([val] is odd) \{
25) vector [floor1\_final\_Y] element [i] =
[predicted] - (([val] + 1) divided by 2 using integer division)