Resources

Handling outliers

A significant result from an outlier test indicates that the observation is unlikely to have occurred by chance. The outlier needs further investigation to try and assign a cause. In general, values should not be removed from a data set solely on the basis of the result of a statistical test. However, there are some guidelines available to assist with handling outliers. ISO 5725 Part 2 (Accuracy (trueness and precision) of measurement methods and results - Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method) contains the following recommendations:

• Carry out the outlier test at the 95% and 99% confidence levels
• Investigate any significant outliers and correct any errors identified
• Reject outliers that are significant at the 99% confidence level unless there is a technical reason to retain them
• Retain outliers that are significant only at the 95% confidence level (sometimes referred to as ‘stragglers’) unless there is a technical reason to reject them.

The standard permits successive testing and rejection of outliers but care should be taken to avoid rejecting a substantial proportion of the data.