Annotation 2D Gesture TNNN_SD.ges This file type contains information on gestures that occured in the session. All gestures that occur within the borders of the SIVIT camera field are labelled. Additionally, emotional gestures that occur elsewhere are labeled. Note that this annotation is completely integrated in the BAS Partitur Format files (BPF) found in 'annot/par' (tier 'GES'). The original *.ges files are stored in 'annot/ges'. 2. column: Begin of the gesture (onset) in samples (1/16 000 Sec) 3. column: Duration of the gesture in samples 4. column: Category of the gesture 5. column: Label of the gesture 6. column: Hand and finger used 7. column: reference word 8. column: reference zone 9. column: reference object 10. column: Begin of the stroke in samples 11. column: Duration of the strok in samples 12. column: Optional: comments about the event Category: The following categories exist: I-gesture [I-Geste]: a request, e.g. pointing, circling U-gesture [U-Geste]: a supporting gesture that is not a request, it prepares a request, e.g. reading, searching R-gesture [R-Geste]: an unidentifiable gesture or an emotional gesture Labels: The following labels exist: 1. long pointing with/without +/- touching the display [I - deut +] [I - deut -] 2. short pointing (19 frames or less) with/without touching the display [I - tipp +] [I - tipp -] 3. circling with/without touching the display [I - kreis +] [I- kreis -] 4. complex gestures within/outside of the display area [I - frei +] [I - frei -] 5. reading/moving hand [U - les - k] 6. searching [U - such - k] 7. counting [U - zähl - k] 8. pondering/moving hand [U - überleg - k] 9. reading/hand not moving [U - les - p] 10. pondering/hand not moving [U - überleg - p] 11.-12. emotional within/without display [R -emot +] [R - emot -] 13. unidentifiable gestures [R - UFO] 14. gesture is not clearly visible [nicht erkennbar] Finger, hand or pen used for the gesture: This string consists of two or three words: '[FINGER] re|li [TOOL]' The (optional) string FINGER denotes one of the five finger types used by the person in the gesture: 'Zeige' = index finger 'Mittel' = middle finger 'Ring' = ring finger 'Kleiner' = small finger If more than one finger or a pen is used, the string [FINGER] is empty. 're' denotes right (German 'rechts'); 'li' denotes left (German 'links') The string [TOOL] is mandatory and is either set to 'Hand' if the gesture is done by the hand (and optional one finger) or set to 'Stift' (German for 'pen') if the gesture is performend using a pen. In the latter case the string [FINGER] is always empty. For instance 'Zeige re Hand' denotes the index finger of the right hand; 'li Hand' denotes the left hand (more than one finger used); 'li Stift' denotes a gesture performed with the left hand holding a pen. If the pointing method is not clearly visible this label might also contain the text [nicht erkennbar]. Reference word: The spoken word that is time-aligned to the gesture. The string may have one of the three forms: 'Word' : the gesture takes place during the utterance of 'Word'. '§ Word' : The gesture is before the utterance of 'Word'. 'Word §' : The gesture takes place after the utterance of 'Word' The reference word is only labelled in I-gestures. Reference zone: Part of the display where gesture takes place (middle or the four corners) Reference object: Describes if and how object was hit or not: 'Treffer' : object was hit 'oberhalb' : gesture marks area above the object 'unterhalb' : gesture marks area below the object 'links' : gesture marks area left of the object 'rechts' : gesture marks area right of the object 'leer' : gesture does not mark any object Begin/Begin Stroke: Onset of the gesture and onset of the stroke = most important part of the gesture are marked separately For more information see 'TechDok-NR-14.ps' in the subdir 'doc/techdocs' (German) or 'doc/pardoc/index.html' (English). You may also refer to the following publications: doc/papers/Steininger-London-01.ps doc/papers/Steininger-Verona-01.pdf