Proficiency testing

Importance of proficiency testing (PT) for analytical quality control

PTs plays an important role in quality monitoring of analytical laboratories. With a laboratory audit it can be checked if there is a basis for good quality of analysis and if the routine procedures in laboratory are sufficient for good quality results. But the actual analytical performance can only be checked by comparing the results in an interlaboratory comparison and with stated assigned values. Therefore the government of Baden-Württemberg asked the Institute for Sanitary Engineering, Water Quality and Solid Waste Management in 1987 to provide PTs for the analysis of waste and ground water. Meanwhile these PTs are also well-known outside of Baden-Württemberg. Successful participation in these PTs is necessary for the governmental permission to analyse waste water according to "Verordnung des Ministeriums für Umwelt und Verkehr über sachverständige Stellen in der Wasserwirtschaft" and for the permission to analyse drinking water in Germany. But many laboratories use these PTs also on a voluntary basis as proof of competence for accreditation bodies and customers.

DIN 38402 - A45 / DIN EN ISO/IEC 17043 / DIN ISO 13528

DIN 38402 - A45 (Sept. 2003) describes details for the provision of PTs for water analysis. It contains requirements for PT provider (e.g. personnel, equipment, QM system) as well as requirements for the concrete planning of PT exercises (PT plan, selection of samples and parameters). Specifications are given for the execution of the PT exercises (preparation of samples, stability and homogeneity testing, measures to prevent collusion, permitted analytical methods, assigned value, sample distribution) and last but not least methods for the statistical evaluation and the assessment are prescribed (Hampel-estimator, Q-method, z_U-scores).

DIN 38402 - A45 is compatible with the requirements of the International Standards EN ISO/IEC 17043:2010 and ISO 13528:2005 and is a concretion for the water sector only.

All PTs provided by AQS Baden-Württemberg are in accordance with the standards and guidelines mentioned above.

Course of action of the PTs from announcement to reporting

Usually the Institute for Sanitary Engineering provides at least 2 PT rounds for waste water, 3 for drinking water, one for the monitoring according to the European Water Framework Directive (together with the PT-WFD network) and one PT for rapid tests in waste water treatment plants. The samples are usually based on a real matrix. Selection of parameters is done in the waste water sector according to the "Fachmodul Wasser" of the "Länderarbeitsgemeinschaft Wasser", in the drinking water sector according to the German Drinking Water Ordinance (based on the European Directive) and for the PT rounds for the Water Framework Directive Monitoring according to the list of Priority Substances. In the PT rounds for the analyses in waste water treatment plants the parameters relevant for the self-monitoring are tested.

The waste water PTs usually are provided as "länderübergreifende Ringversuche". In this case the PT is organised and provided by several PT providers around Germany in an identical way according to agreements in the respective group within the "Länderarbeitsgemeinschaft Wasser". Execution and evaluation are done according to the technical bulletin A-3 of the "Länderarbeitsgemeinschaft Wasser" (LAWA) (AQS-Merkblätter für die Wasser-, Abwasser und Schlammuntersuchung; Erich-Schmidt-Verlag, Berlin).

Drinking water PTs are provided in cooperation with the "Behörde für Soziales, Familie, Gesundheit und Verbraucherschutz" in Hamburg.

The PT rounds for the monitoring according to the Water Framework Directive are provided within the PT-WFD network in cooperation with other European PT providers. Please find more information on the webpage of the PT-WFD network.

The PT rounds for rapid test analysis in waster water treatment plants are organised on behalf of the Ministry of the Environment, Nature Conservation and Transport Baden-Württemberg.

AQS Baden-Württemberg at the ISWA also organises interlaboratory comparisons for other customers on request, e.g. in the framework of the Ground Water Quality Monitoring Network in Baden-Württemberg for the "Landesanstalt für Umwelt, Messungen und Naturschutz". In this monitoring programme the successful participation in such interlaboratory comparisons is a precondition to get a contract for analyses for certain parameters.

Participants get three similar samples, which contain the various components in different concentrations. AQS-BW determines the necessary stabilization measures, decides about type of containers and concentration levels, if necessary after pre-investigations and consultation of other PT providers involved. The lowest concentration is chosen in such a way that it can be accurately measured with at least one of methods described in the relevant standards. The concentration range is chosen so that a relevant limit is included in the lower part of the range and the highest concentration is still in the region that can be found in reality.

The PT exercise is announced to all registered laboratories some weeks before the planned dispatch date together with a list of suitable/prescribed methods. A deadline for registration is given for this exercise.

In the Institute for Sanitary Engineering stock solutions for all parameters and all concentration levels are prepared with high effort for controlling. Via a fixed system of dilution steps samples are prepared from these stock solutions.

The amount of sample is fixed in a way to ensure that on one hand it will be suitable for multiple determinations, but on the other hand it is so low such as not to allow excessive effort for the analysis of the PT samples exceeding the normal routine effort.

The samples are packed into cardboxes using PU-mouldings and dispatched using an express parcel service. In most cases the parcels arrive in the participating laboratory in the following day, so that analyses can start in the same week. In some cases, where an uninterrupted cooling has to be ensured, the samples are brought to distribution points with cooling trucks and distributed to the laboratories directly.

The parcels also contain a leaflet, pointing at some frequent mistakes. This leaflet should be used as a checklist before reporting the results.

Reporting of the results can be done either electronically via the Internet or using the paper forms distributed with the samples. If the methods used are reported, there is the possibility to perform a method specific evaluation. The result forms and the printed esult sheets from the internet data input have to be sent back to the Institute for Sanitary Engineering signed and absolutely on time.

For the organization of such a PT it is absolutely necessary to meet all deadlines. Without that samples can not be prepared in the necessary number. Because of organisational reasons sometimes there are some samples left for latecomers. But of course this cannot be guaranteed. The time available for data evaluation is also very short, so that AQS-BW has to insist on timely delivered results.

After evaluation the PT provider prepares data evaluation sheets for all participants and distributes themwith the evaluation documentation. These data evaluation sheets contain all results reported by the participants and all assigned values. Values inside the tolerance limits are marked as successful with a "+".

Evaluation and Assessment

The PT exercises of AQS Baden-Württemberg are evaluated according to DIN 38402 - A45. The assessment uses the deviation of the result from the assigned value expressed as z_U-score.

The first step is the calculation of the reproducibility standard deviation s_R for all concentration levels. This is done using the Q-method, a robust statistical method. This value is also necessary for the calculation of the Hampel-estimator, a robust mean. Generally this robust mean is used as conventional true value, the assigned value. In exceptional cases the PT coordinator can deviate from this rule and define the assigned value using reference values from weighings or other sources.

In a first step z-scores are calculated for all measurement result using the following formula:

z-score = (result - assigned value) / standard deviation for proficiency assessment

The standard deviation for proficiency assessment may deviate from the calculated reproducibility standard deviation, if
a)upper and lower limits were fixed in advance and these limits were exceeded. In this case the exceeded limit is used as standard deviation for proficiency assessment
or if
b) a common assessment for all concentration levels using the variance function described in DIN 38402 - A45 was done. In this case the value of the function at the respective concentration is used as standard deviation for proficiency assessment.
A combination of both cases is also possible.

To avoid injustice in the assessment due to a skewed statistical distribution of values at low concentrations z_U-scores are calculated from the z-scores as described in DIN 38402-A45 (or ISO/TS 20612).

All measurement results are assessed according to their z_U-scores. Values between -2 and +2 are acceptable by definition. More deviating values are assessed as "wrong". In the PT exercises of the PT-WFD network values in the range 2 < |z-score| < 3 are assessed as "questionable".

The assessment of all values are noted on the data evaluation sheet with "+" or "-".

The further parameter specific assessment and a total assessment for the PT round is different in the waste water and drinking water sector and therefore is described on the respective webpages for these PT rounds.