This article needs additional citations for verification. (September 2016) |
Observational error (or measurement error) is the difference between a measured value of a quantity and its unknown true value. Such errors are inherent in the measurement process; for example lengths measured with a ruler calibrated in whole centimeters will have a measurement error of several millimeters. The error or uncertainty of a measurement can be estimated, and is specified with the measurement as, for example, 32.3 ± 0.5 cm.
Scientific observations are marred by two distinct types of errors, systematic errors on the one hand, and random, on the other hand. The effects of random errors can be mitigated by the repeated measurements. Constant or systematic errors on the contrary must be carefully avoided, because they arise from one or more causes which constantly act in the same way, and have the effect of always altering the result of the experiment in the same direction. They therefore alter the value observed and repeated identical measurements do not reduce such errors.
Measurement errors can be summarized in terms of accuracy and precision. For example, length measurements with a ruler accurately calibrated in whole centimeters will be subject to random error with each use on the same distance giving a slightly different value resulting limited precision; a metallic ruler the temperature of which is not controled will be affected by thermal expansion causing an additional systematic error resulting in limited accuracy.
Science and experiments
When either randomness or uncertainty modeled by probability theory is attributed to such errors, they are "errors" in the sense in which that term is used in statistics; see errors and residuals in statistics.
Every time a measurement is repeated, slightly different results are obtained. The common statistical model used is that the error has two additive parts:
- Random error which may vary from observation to another.
- Systematic error which always occurs, with the same value, when we use the instrument in the same way and in the same case.
Some errors are not clearly random or systematic such as the uncertainty in the calibration of an instrument.
Random errors or statistical errors in measurement lead to measurable values being inconsistent between repeated measurements of a constant attribute or quantity are taken. Random errors create measurement uncertainty. These error are uncorrelated between measurements. Repeated measurements will fall in a pattern and in a large set of such measurements a standard deviation can be calculated as a estimate of the amount of statistical error.: 147
Systematic errors are errors that are not determined by chance but are introduced by repeatable processes inherent to the system. Sources of systematic errors include errors in equipment calibration, uncertainty in correction terms applied during experimental analysis, errors due the use of approximate theoretical models.: supl Systematic error is sometimes called statistical bias. It may often be reduced with standardized procedures.
Part of the learning process in the various sciences is learning how to use standard instruments and protocols so as to minimize systematic error. Over a long period of time, systematic errors in science can be resolved and become a form of "negative knowledge": scientist build up an understanding of how to avoid specific kinds of systematic errors.
Propagation of errors
When two or more observations or two or more instruments are combined, the errors in each combine. Estimates of the error in the result of such combinations depend upon the statistical characteristics of each individual measurement and on the possible statistical correlation between them.: 92
Characterization
Measurement errors can be divided into two components: random error and systematic error.
Random error is always present in a measurement. It is caused by inherently unpredictable fluctuations in the readings of a measurement apparatus or in the experimenter's interpretation of the instrumental reading. Random errors show up as different results for ostensibly the same repeated measurement. They can be estimated by comparing multiple measurements and reduced by averaging multiple measurements.
Systematic error is predictable and typically constant or proportional to the true value. If the cause of the systematic error can be identified, then it usually can be eliminated. Systematic errors are caused by imperfect calibration of measurement instruments or imperfect methods of observation, or interference of the environment with the measurement process, and always affect the results of an experiment in a predictable direction. Incorrect zeroing of an instrument is an example of systematic error in instrumentation.
The Performance Test Standard PTC 19.1-2005 "Test Uncertainty", published by the American Society of Mechanical Engineers (ASME), discusses systematic and random errors in considerable detail. In fact, it conceptualizes its basic uncertainty categories in these terms.
Random error can be caused by unpredictable fluctuations in the readings of a measurement apparatus, or in the experimenter's interpretation of the instrumental reading; these fluctuations may be in part due to interference of the environment with the measurement process. The concept of random error is closely related to the concept of precision. The higher the precision of a measurement instrument, the smaller the variability (standard deviation) of the fluctuations in its readings.
Sources
Sources of systematic error
Imperfect calibration
Sources of systematic error may be imperfect calibration of measurement instruments (zero error), changes in the environment which interfere with the measurement process and sometimes imperfect methods of observation can be either zero error or percentage error. If you consider an experimenter taking a reading of the time period of a pendulum swinging past a fiducial marker: If their stop-watch or timer starts with 1 second on the clock then all of their results will be off by 1 second (zero error). If the experimenter repeats this experiment twenty times (starting at 1 second each time), then there will be a percentage error in the calculated average of their results; the final result will be slightly larger than the true period.
Distance measured by radar will be systematically overestimated if the slight slowing down of the waves in air is not accounted for. Incorrect zeroing of an instrument is an example of systematic error in instrumentation.
Systematic errors may also be present in the result of an estimate based upon a mathematical model or physical law. For instance, the estimated oscillation frequency of a pendulum will be systematically in error if slight movement of the support is not accounted for.
Quantity
Systematic errors can be either constant, or related (e.g. proportional or a percentage) to the actual value of the measured quantity, or even to the value of a different quantity (the reading of a ruler can be affected by environmental temperature). When it is constant, it is simply due to incorrect zeroing of the instrument. When it is not constant, it can change its sign. For instance, if a thermometer is affected by a proportional systematic error equal to 2% of the actual temperature, and the actual temperature is 200°, 0°, or −100°, the measured temperature will be 204° (systematic error = +4°), 0° (null systematic error) or −102° (systematic error = −2°), respectively. Thus the temperature will be overestimated when it will be above zero and underestimated when it will be below zero.
Drift
Systematic errors which change during an experiment (drift) are easier to detect. Measurements indicate trends with time rather than varying randomly about a mean. Drift is evident if a measurement of a constant quantity is repeated several times and the measurements drift one way during the experiment. If the next measurement is higher than the previous measurement as may occur if an instrument becomes warmer during the experiment then the measured quantity is variable and it is possible to detect a drift by checking the zero reading during the experiment as well as at the start of the experiment (indeed, the zero reading is a measurement of a constant quantity). If the zero reading is consistently above or below zero, a systematic error is present. If this cannot be eliminated, potentially by resetting the instrument immediately before the experiment then it needs to be allowed by subtracting its (possibly time-varying) value from the readings, and by taking it into account while assessing the accuracy of the measurement.
If no pattern in a series of repeated measurements is evident, the presence of fixed systematic errors can only be found if the measurements are checked, either by measuring a known quantity or by comparing the readings with readings made using a different apparatus, known to be more accurate. For example, if you think of the timing of a pendulum using an accurate stopwatch several times you are given readings randomly distributed about the mean. Hopings systematic error is present if the stopwatch is checked against the 'speaking clock' of the telephone system and found to be running slow or fast. Clearly, the pendulum timings need to be corrected according to how fast or slow the stopwatch was found to be running.
Measuring instruments such as ammeters and voltmeters need to be checked periodically against known standards.
Systematic errors can also be detected by measuring already known quantities. For example, a spectrometer fitted with a diffraction grating may be checked by using it to measure the wavelength of the D-lines of the sodium electromagnetic spectrum which are at 600 nm and 589.6 nm. The measurements may be used to determine the number of lines per millimetre of the diffraction grating, which can then be used to measure the wavelength of any other spectral line.
Constant systematic errors are very difficult to deal with as their effects are only observable if they can be removed. Such errors cannot be removed by repeating measurements or averaging large numbers of results. A common method to remove systematic error is through calibration of the measurement instrument.
Sources of random error
The random or stochastic error in a measurement is the error that is random from one measurement to the next. Stochastic errors tend to be normally distributed when the stochastic error is the sum of many independent random errors because of the central limit theorem. Stochastic errors added to a regression equation account for the variation in Y that cannot be explained by the included Xs.
Surveys
The term "observational error" is also sometimes used to refer to response errors and some other types of non-sampling error. In survey-type situations, these errors can be mistakes in the collection of data, including both the incorrect recording of a response and the correct recording of a respondent's inaccurate response. These sources of non-sampling error are discussed in Salant and Dillman (1994) and Bland and Altman (1996).
These errors can be random or systematic. Random errors are caused by unintended mistakes by respondents, interviewers and/or coders. Systematic error can occur if there is a systematic reaction of the respondents to the method used to formulate the survey question. Thus, the exact formulation of a survey question is crucial, since it affects the level of measurement error. Different tools are available for the researchers to help them decide about this exact formulation of their questions, for instance estimating the quality of a question using MTMM experiments. This information about the quality can also be used in order to correct for measurement error.
Effect on regression analysis
If the dependent variable in a regression is measured with error, regression analysis and associated hypothesis testing are unaffected, except that the R2 will be lower than it would be with perfect measurement.
However, if one or more independent variables is measured with error, then the regression coefficients and standard hypothesis tests are invalid. This is known as attenuation bias.
See also
- Bias (statistics)
- Cognitive bias
- Correction for measurement error (for Pearson correlations)
- Error
- Errors and residuals in statistics
- Errors-in-variables models
- Instrument error
- Measurement uncertainty
- Metrology
- Outlier
- Propagation of uncertainty
- Regression dilution
- Replication (statistics)
- Statistical theory
- Systemic bias
- Test method
References
- Dodge, Y. (2003) The Oxford Dictionary of Statistical Terms, OUP. ISBN 978-0-19-920613-1
- John Robert Taylor (1999). An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements. University Science Books. p. 94, §4.1. ISBN 978-0-935702-75-0.
- Ritter, Elie. Manuel théorique et pratique de l'application de la méthode des moindres carrés au calcul des observations. Mallet-Bachelier. p. 7. Retrieved 16 February 2025.
- Heinrich, Joel; Lyons, Louis (2007-11-01). "Systematic Errors". Annual Review of Nuclear and Particle Science. 57 (1): 145–169. doi:10.1146/annurev.nucl.57.090506.123052. ISSN 0163-8998.
- "Systematic error". Merriam-webster.com. Retrieved 2016-09-10.
- Allchin, Douglas (March 2001). "Error Types". Perspectives on Science. 9 (1): 38–58. doi:10.1162/10636140152947786. ISSN 1063-6145.
- Young, Hugh D. (1996). Statistical treatment of experimental data: an introduction to statistical methods (Repr ed.). Long Grove, Ill: Waveland Press. ISBN 978-0-88133-913-0.
- Salant, P.; Dillman, D. A. (1994). How to conduct your survey. New York: John Wiley & Sons. ISBN 0-471-01273-4.
- Bland, J. Martin; Altman, Douglas G. (1996). "Statistics Notes: Measurement Error". BMJ. 313 (7059): 744. doi:10.1136/bmj.313.7059.744. PMC 2352101. PMID 8819450.
- Saris, W. E.; Gallhofer, I. N. (2014). Design, Evaluation and Analysis of Questionnaires for Survey Research (Second ed.). Hoboken: Wiley. ISBN 978-1-118-63461-5.
- DeCastellarnau, A. and Saris, W. E. (2014). A simple procedure to correct for measurement errors in survey research. European Social Survey Education Net (ESS EduNet). Available at: http://essedunet.nsd.uib.no/cms/topics/measurement Archived 2019-09-15 at the Wayback Machine
- Saris, W. E.; Revilla, M. (2015). "Correction for measurement errors in survey research: necessary and possible" (PDF). Social Indicators Research. 127 (3): 1005–1020. doi:10.1007/s11205-015-1002-x. hdl:10230/28341. S2CID 146550566.
- Hayashi, Fumio (2000). Econometrics. Princeton University Press. p. 187. ISBN 978-0-691-01018-2.
- Angrist, Joshua David; Pischke, Jörn-Steffen (2015). Mastering 'metrics : the path from cause to effect. Princeton, New Jersey: Princeton University Press. p. 221. ISBN 978-0-691-15283-7. OCLC 877846199.
The bias generated by this sort of measurement error in regressors is called attenuation bias.
Further reading
- Cochran, W. G. (1968). "Errors of Measurement in Statistics". Technometrics. 10 (4): 637–666. doi:10.2307/1267450. JSTOR 1267450. S2CID 120645541.
This article needs additional citations for verification Please help improve this article by adding citations to reliable sources Unsourced material may be challenged and removed Find sources Observational error news newspapers books scholar JSTOR September 2016 Learn how and when to remove this message Observational error or measurement error is the difference between a measured value of a quantity and its unknown true value Such errors are inherent in the measurement process for example lengths measured with a ruler calibrated in whole centimeters will have a measurement error of several millimeters The error or uncertainty of a measurement can be estimated and is specified with the measurement as for example 32 3 0 5 cm Scientific observations are marred by two distinct types of errors systematic errors on the one hand and random on the other hand The effects of random errors can be mitigated by the repeated measurements Constant or systematic errors on the contrary must be carefully avoided because they arise from one or more causes which constantly act in the same way and have the effect of always altering the result of the experiment in the same direction They therefore alter the value observed and repeated identical measurements do not reduce such errors Measurement errors can be summarized in terms of accuracy and precision For example length measurements with a ruler accurately calibrated in whole centimeters will be subject to random error with each use on the same distance giving a slightly different value resulting limited precision a metallic ruler the temperature of which is not controled will be affected by thermal expansion causing an additional systematic error resulting in limited accuracy Science and experimentsWhen either randomness or uncertainty modeled by probability theory is attributed to such errors they are errors in the sense in which that term is used in statistics see errors and residuals in statistics Distribution of measurements of known true value with both constant systematic error and normally distributed random error Every time a measurement is repeated slightly different results are obtained The common statistical model used is that the error has two additive parts Random error which may vary from observation to another Systematic error which always occurs with the same value when we use the instrument in the same way and in the same case Some errors are not clearly random or systematic such as the uncertainty in the calibration of an instrument Random errors or statistical errors in measurement lead to measurable values being inconsistent between repeated measurements of a constant attribute or quantity are taken Random errors create measurement uncertainty These error are uncorrelated between measurements Repeated measurements will fall in a pattern and in a large set of such measurements a standard deviation can be calculated as a estimate of the amount of statistical error 147 Systematic errors are errors that are not determined by chance but are introduced by repeatable processes inherent to the system Sources of systematic errors include errors in equipment calibration uncertainty in correction terms applied during experimental analysis errors due the use of approximate theoretical models supl Systematic error is sometimes called statistical bias It may often be reduced with standardized procedures Part of the learning process in the various sciences is learning how to use standard instruments and protocols so as to minimize systematic error Over a long period of time systematic errors in science can be resolved and become a form of negative knowledge scientist build up an understanding of how to avoid specific kinds of systematic errors Propagation of errorsWhen two or more observations or two or more instruments are combined the errors in each combine Estimates of the error in the result of such combinations depend upon the statistical characteristics of each individual measurement and on the possible statistical correlation between them 92 CharacterizationMeasurement errors can be divided into two components random error and systematic error Random error is always present in a measurement It is caused by inherently unpredictable fluctuations in the readings of a measurement apparatus or in the experimenter s interpretation of the instrumental reading Random errors show up as different results for ostensibly the same repeated measurement They can be estimated by comparing multiple measurements and reduced by averaging multiple measurements Systematic error is predictable and typically constant or proportional to the true value If the cause of the systematic error can be identified then it usually can be eliminated Systematic errors are caused by imperfect calibration of measurement instruments or imperfect methods of observation or interference of the environment with the measurement process and always affect the results of an experiment in a predictable direction Incorrect zeroing of an instrument is an example of systematic error in instrumentation The Performance Test Standard PTC 19 1 2005 Test Uncertainty published by the American Society of Mechanical Engineers ASME discusses systematic and random errors in considerable detail In fact it conceptualizes its basic uncertainty categories in these terms Random error can be caused by unpredictable fluctuations in the readings of a measurement apparatus or in the experimenter s interpretation of the instrumental reading these fluctuations may be in part due to interference of the environment with the measurement process The concept of random error is closely related to the concept of precision The higher the precision of a measurement instrument the smaller the variability standard deviation of the fluctuations in its readings SourcesSources of systematic error Imperfect calibration Sources of systematic error may be imperfect calibration of measurement instruments zero error changes in the environment which interfere with the measurement process and sometimes imperfect methods of observation can be either zero error or percentage error If you consider an experimenter taking a reading of the time period of a pendulum swinging past a fiducial marker If their stop watch or timer starts with 1 second on the clock then all of their results will be off by 1 second zero error If the experimenter repeats this experiment twenty times starting at 1 second each time then there will be a percentage error in the calculated average of their results the final result will be slightly larger than the true period Distance measured by radar will be systematically overestimated if the slight slowing down of the waves in air is not accounted for Incorrect zeroing of an instrument is an example of systematic error in instrumentation Systematic errors may also be present in the result of an estimate based upon a mathematical model or physical law For instance the estimated oscillation frequency of a pendulum will be systematically in error if slight movement of the support is not accounted for Quantity Systematic errors can be either constant or related e g proportional or a percentage to the actual value of the measured quantity or even to the value of a different quantity the reading of a ruler can be affected by environmental temperature When it is constant it is simply due to incorrect zeroing of the instrument When it is not constant it can change its sign For instance if a thermometer is affected by a proportional systematic error equal to 2 of the actual temperature and the actual temperature is 200 0 or 100 the measured temperature will be 204 systematic error 4 0 null systematic error or 102 systematic error 2 respectively Thus the temperature will be overestimated when it will be above zero and underestimated when it will be below zero Drift Systematic errors which change during an experiment drift are easier to detect Measurements indicate trends with time rather than varying randomly about a mean Drift is evident if a measurement of a constant quantity is repeated several times and the measurements drift one way during the experiment If the next measurement is higher than the previous measurement as may occur if an instrument becomes warmer during the experiment then the measured quantity is variable and it is possible to detect a drift by checking the zero reading during the experiment as well as at the start of the experiment indeed the zero reading is a measurement of a constant quantity If the zero reading is consistently above or below zero a systematic error is present If this cannot be eliminated potentially by resetting the instrument immediately before the experiment then it needs to be allowed by subtracting its possibly time varying value from the readings and by taking it into account while assessing the accuracy of the measurement If no pattern in a series of repeated measurements is evident the presence of fixed systematic errors can only be found if the measurements are checked either by measuring a known quantity or by comparing the readings with readings made using a different apparatus known to be more accurate For example if you think of the timing of a pendulum using an accurate stopwatch several times you are given readings randomly distributed about the mean Hopings systematic error is present if the stopwatch is checked against the speaking clock of the telephone system and found to be running slow or fast Clearly the pendulum timings need to be corrected according to how fast or slow the stopwatch was found to be running Measuring instruments such as ammeters and voltmeters need to be checked periodically against known standards Systematic errors can also be detected by measuring already known quantities For example a spectrometer fitted with a diffraction grating may be checked by using it to measure the wavelength of the D lines of the sodium electromagnetic spectrum which are at 600 nm and 589 6 nm The measurements may be used to determine the number of lines per millimetre of the diffraction grating which can then be used to measure the wavelength of any other spectral line Constant systematic errors are very difficult to deal with as their effects are only observable if they can be removed Such errors cannot be removed by repeating measurements or averaging large numbers of results A common method to remove systematic error is through calibration of the measurement instrument Sources of random error The random or stochastic error in a measurement is the error that is random from one measurement to the next Stochastic errors tend to be normally distributed when the stochastic error is the sum of many independent random errors because of the central limit theorem Stochastic errors added to a regression equation account for the variation in Y that cannot be explained by the included Xs SurveysThe term observational error is also sometimes used to refer to response errors and some other types of non sampling error In survey type situations these errors can be mistakes in the collection of data including both the incorrect recording of a response and the correct recording of a respondent s inaccurate response These sources of non sampling error are discussed in Salant and Dillman 1994 and Bland and Altman 1996 These errors can be random or systematic Random errors are caused by unintended mistakes by respondents interviewers and or coders Systematic error can occur if there is a systematic reaction of the respondents to the method used to formulate the survey question Thus the exact formulation of a survey question is crucial since it affects the level of measurement error Different tools are available for the researchers to help them decide about this exact formulation of their questions for instance estimating the quality of a question using MTMM experiments This information about the quality can also be used in order to correct for measurement error Effect on regression analysisIf the dependent variable in a regression is measured with error regression analysis and associated hypothesis testing are unaffected except that the R2 will be lower than it would be with perfect measurement However if one or more independent variables is measured with error then the regression coefficients and standard hypothesis tests are invalid This is known as attenuation bias See alsoBias statistics Cognitive bias Correction for measurement error for Pearson correlations Error Errors and residuals in statistics Errors in variables models Instrument error Measurement uncertainty Metrology Outlier Propagation of uncertainty Regression dilution Replication statistics Statistical theory Systemic bias Test methodReferencesDodge Y 2003 The Oxford Dictionary of Statistical Terms OUP ISBN 978 0 19 920613 1 John Robert Taylor 1999 An Introduction to Error Analysis The Study of Uncertainties in Physical Measurements University Science Books p 94 4 1 ISBN 978 0 935702 75 0 Ritter Elie Manuel theorique et pratique de l application de la methode des moindres carres au calcul des observations Mallet Bachelier p 7 Retrieved 16 February 2025 Heinrich Joel Lyons Louis 2007 11 01 Systematic Errors Annual Review of Nuclear and Particle Science 57 1 145 169 doi 10 1146 annurev nucl 57 090506 123052 ISSN 0163 8998 Systematic error Merriam webster com Retrieved 2016 09 10 Allchin Douglas March 2001 Error Types Perspectives on Science 9 1 38 58 doi 10 1162 10636140152947786 ISSN 1063 6145 Young Hugh D 1996 Statistical treatment of experimental data an introduction to statistical methods Repr ed Long Grove Ill Waveland Press ISBN 978 0 88133 913 0 Salant P Dillman D A 1994 How to conduct your survey New York John Wiley amp Sons ISBN 0 471 01273 4 Bland J Martin Altman Douglas G 1996 Statistics Notes Measurement Error BMJ 313 7059 744 doi 10 1136 bmj 313 7059 744 PMC 2352101 PMID 8819450 Saris W E Gallhofer I N 2014 Design Evaluation and Analysis of Questionnaires for Survey Research Second ed Hoboken Wiley ISBN 978 1 118 63461 5 DeCastellarnau A and Saris W E 2014 A simple procedure to correct for measurement errors in survey research European Social Survey Education Net ESS EduNet Available at http essedunet nsd uib no cms topics measurement Archived 2019 09 15 at the Wayback Machine Saris W E Revilla M 2015 Correction for measurement errors in survey research necessary and possible PDF Social Indicators Research 127 3 1005 1020 doi 10 1007 s11205 015 1002 x hdl 10230 28341 S2CID 146550566 Hayashi Fumio 2000 Econometrics Princeton University Press p 187 ISBN 978 0 691 01018 2 Angrist Joshua David Pischke Jorn Steffen 2015 Mastering metrics the path from cause to effect Princeton New Jersey Princeton University Press p 221 ISBN 978 0 691 15283 7 OCLC 877846199 The bias generated by this sort of measurement error in regressors is called attenuation bias Further readingCochran W G 1968 Errors of Measurement in Statistics Technometrics 10 4 637 666 doi 10 2307 1267450 JSTOR 1267450 S2CID 120645541
