Manual Validation

This chapter describes those validations of the corpus data that cannot be performed automatically and therefore require considerable efforts in terms of manpower. Typically, this will be carried out by a group of selected validators - usually with special skills and native speakers of the corpus language - under the supervision of a person responsible for the logistics. This supervisor must be able to

• select (maybe also train) the validators
• organize the selection of validation material
• organize the distribution of work load to the validators
• collect and check the results
• do quality control (e.g. by taking random sample checks)
• calculate the results (some basic statistics)

Since the variety of annotations is large, we cannot give detailed advice on how to validate all the different annotations schemes. Instead we will concentrate on some practical hints that will most likely be useful in all kinds of manual corpus validations.