Understanding DRO Scale Parameters

When building a DRO the selection of scales is the most critical decision. TouchDRO can work with a variety DRO scales, ranging from inexpensive calipers to sub-micron glass scales. In addition to the cost considerations there are technical parameters that you should take into account.

Scale Parameter Overview

There are a few factors that should drive your decision. In addition to the cost and availability, such things as resolution, accuracy and several other factors need to be taken into account. Unfortunately many manufacturers intentionally or unintentionally confuse these parameters to make their scales look better, so it's important to understand what each parameter means.

Resolution

Picture of measuring devices with different resolutons
Fig 1: Measurement Instruments with Different Resolutions

Resolution determines how fine the measurements will be, or for the purposes for a DRO, how fine are the divisions on the encoder. For example, the instruments above have resolutions of 1/16", 1/100", 1/1,000", and 1/10,000" respectively.

Accuracy

Accuracy is often confused with resolution but it's a very different parameter. Accuracy tells you how close the reading from the scale will match the actual dimension. For example, when you measure a 1" standard, an accurate scale will read exactly one inch, while a bad one will show a different value. It's possible to have very high resolution but very poor accuracy. For instance, grinding off some metal off the micrometer anvil doesn't affect the resolution but it won't be at all accurate.

Side note: some manufacturers/retailers often use the term "accuracy" to encompass such parameters as repeatability, hysteresis and runout, while others confuse it with resolution.

Repeatability

Repeatability defines how close together a set of successive readings will be to each other. For example, it's possible to affect the readings from a micrometer by applying different amounts of torque, so the readings won't be repeatable (unless I use the clutch properly). In other words, when you move a scale by exactly one inch several times (from the same starting position), a repeatable scale will provide readings that are identical or very close together.

Hysteresis

Hysteresis is the amount of travel the reading head can move before the scale starts registering the change in position and can be thought of as scale's backlash. To measure the hysteresis, first move the scale one way by exactly one inch in one direction, then by the same amount in the opposite direction, and compare the readout.

Runout

Runout is another aspect of the overall accuracy that is as important as the repeatability. In a nutshell it describes how uniform the resolution of the scales is. For instance, when you move the scale from one end to the other in exactly one inch increments a good scale will give you a series of identical readings. You can think of runout as deviation in resolution over the scales length. If there is any difference it will lead to an error of measurement that will accumulate over longer distances.

Reliability

Reliability is a somewhat subjective value and depends to a large degree of the shop environment, usage etc. Every scale will fail under some conditions, so it's up to you to determine what your particular usage envelope will be and if the scales are fit for it. Nevertheless some scales will be more reliable and last longer as a result of better protection, higher quality materials etc.

Reading Speed

Reading speed, sometimes also called "refresh rate" determines how often the scale updates its position. This parameter is usually not documented by the scale manufacturers but is extremely important for optimal DRO performance.

Practical Implications

All of the above parameters will influence the overall performance of the DRO. While it's easy to become obsessed with the resolution of the scale, it's only one of the key factors you should look at. In practice even inexpensive calipers provide adequate resolution for most application; 5 micron resolution (~0.0002") is better than vast majority of milling machines and lathes can hold in a climate controller shop, and 1 micron scales are an overkill even for an industrial surface grinder.

Instead you should concentrate on the repeatability, as it is more important, as it determines the overall performance of a DRO. For instance, 0.0001" resolution is pretty much worthless when the scale is only repeatable to 0.005". As long as the scale is repeatable, the software can easily compensate for the linear error.
Consider the illustration below; the first two cases will work equally well in a DRO, since the application will compensate for the offset. Third case is less precise but will get you in the "ball park". The last case, though, is pretty much worthless for any precision work as the readings will be all over the place.

Fig 2: Accuracy vs. Repeatability

Second critical property is the runout. Similar to the repeatability, runout determines the accuracy of DRO's position. Imagine that you need to drill a row of holes exactly 1.000" apart. If a scale has high runout, the holes will not be spaced equally, as shown in the figure below.

Fig 3: Effect of DRO Scale Runout

Third important characteristic is the hysteresis, which will determine how much "backlash" the DRO will have. Most glass and magnetic scales have very small hysteresis, and the parameter is usually listed on the specifications sheet. capacitive scales, and especially inexpensive calipers can have rather large backlash, but the manufacturers don't list this parameter, unfortunately.

Fig 4: Effect of Hysteresis

Finally, refresh rate should not be overlooked. If the refresh rate is too low there will be perceivable lag between the machine's movement and the DRO display. A refresh rate of 20Hz or above feels pretty much "real-time", while 5Hz and less is noticeably laggy, resulting in overshoots.