# A mathematical framework for Transformer Circuits

Source: https://www.anthropic.com/research/a-mathematical-framework-for-transformer-circuits
Published: 2021-12-22
Status: unreviewed source report

## Source excerpt (up to 25 words)

> Reverse-engineering small attention-only transformers reveals how induction heads form via head composition.

## Review question

Check the problem size, uniformity, resource measure, reduction and approximation factor. A claim in one restricted model should not be promoted to a statement about every algorithm.

No independent proof check or claim of peer review.