How does the accuracy of numbers entered into the VCS work?

Question

In a Sound I was looking at the fractional and integer parts of a value entered through the VCS and I was getting some results I didn't fully understand. 

I've boiled it down to this simple example using two STGCs, one showing the integer part and one the fractional part of a value entered:

 

 

The value of the integer STGC is this:

 

and the value of the fraction STGC is this:

 

If I set the range of the Input fader to [-10 10], then I get this, which is what I was expecting:

 

But if I set the range of the input VCS fader to [-100 100] I get this, which I wasn't expecting:

 

If I keep the range the same, but change the Grid to 1.0 I get the first result again. If I change the Grid to 0.1 I get:

 

(Also if I set a small range, [-10 10] and save a preset with the input value of 1.0, then change the range to [-100 100] and recall the preset, I get the first result (integer=1 fraction=0) not the second result.)

If I change the fader to a smallfader I get the same results so I'm assuming it's got nothing to do with the pixel resolution on the screen determining the accuracy of fader positions. And then if I change the range to a large binary number eg. [-65536 65536] I get something like the first result:

 

Also with this range set if I type in 10.1 as the input value the VCS changes like so:

Where the Input and InputFraction both seem like their display is truncated to 5 significant digits and so look kind of different. 

 

What's the best way to think about integers and fractions in the VCS and ranges of faders?

 

Answer 1

VCS values are shown to about 4 significant digits. To see exact results, set the range of the fader to be a power of 2.

When you ask for ranges that are powers-of-10, rescaling them to the [0,1] range results in a repeating binary fraction which is either slightly larger or slightly smaller than the number you asked for (hence the different numbers displayed in the number field for your two different fader ranges, when the display is limited to 4 significant digits).

In your [-10,10] range example, when you enter the value, 1 in the VCS, it converts the value to a [0,1] range which would be

value - min / (max - min)

or

1 - (-10) / (10 - (-10) which is 11/20. This is a repeating binary fraction that is slightly bigger than 1. The binary fraction is sent to the Pacarana. To display the result in the STGC, the Pacarana multiplies by the fader range and adds the minimum to turn it back into the range you asked for. Because it was a repeating binary fraction, the number will either be a little over or a little under what you asked for.