Skip to content
Text as Data and NLP for Finance / Embeddings and classification

LLM Annotation

Supervised classification needs labels. A large language model can be prompted to emit those labels with no task-specific training, which is the zero-shot setup. The measurement decision is still the codebook: relevance, stance, topic, or tone means whatever the instructions say. The implementation is a prompt plus a model checkpoint, used beside human coders. The validation deliverable is accuracy and agreement against people on the same items. These sources do not study 10-K tone as a return predictor. This note does not treat a model label as a finance alpha.
Learning objectives. By the end of this note you should be able to:
  • define zero-shot annotation and state Gilardi, Alizadeh, and Kubli’s comparison with crowd-workers on their four datasets;
  • state the Ziems et al. limit: prompted models did not, except in minority cases, match carefully fine-tuned classifiers, and were often too weak to replace human annotation;
  • specify a blended human–model workflow for a disclosure-tone codebook;
  • list the claims this extract refuses.

In this lesson

  1. What the annotation papers measured
  2. Implementation for a disclosure codebook
  3. Claims these sources do not support
  4. Design choices that the papers actually vary

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Text as Data and NLP for Finance syllabus.