Shakespeare’s Sonnets

A whole-corpus computational reading: meter, rhyme, ambiguity, and learned weights

The 1609 Quarto — 154 sonnets, 2,155 lines — is small enough to read and large enough to count. This page does both: every table and figure below is computed by the code shown, on the corpus that ships with Prosodic (corpora/corppoetry_en/en.shakespeare.txt). To rerun it yourself, install Prosodic and execute the cells top to bottom; the site itself embeds frozen results (quarto’s freeze), so what you see is one real execution, not a mock-up.

Load and parse

The parser is exhaustive: for every line it evaluates every segmentation into weak/strong positions against the constraint set, then discards candidates that lose under any possible weighting (harmonic bounding — see Metrical parsing). Vectorization makes this affordable: the whole corpus parses in about two seconds on a laptop CPU.

import time, warnings
from pathlib import Path
warnings.filterwarnings('ignore')

import pandas as pd
import prosodic

# resolve the bundled corpus relative to wherever this page is executed;
# fall back to the GitHub copy if run outside a prosodic checkout
_rel = Path('corpora') / 'corppoetry_en' / 'en.shakespeare.txt'
_root = next((p for p in [Path.cwd(), *Path.cwd().parents]
              if (p / _rel).exists()), None)
corpus_fn = (str(_root / _rel) if _root else
             'https://raw.githubusercontent.com/quadrismegistus/prosodic'
             '/master/corpora/corppoetry_en/en.shakespeare.txt')

t0 = time.perf_counter()
corpus = prosodic.Text(fn=corpus_fn)
t1 = time.perf_counter()
corpus.parse()
t2 = time.perf_counter()

print(f"{len(corpus.stanzas)} sonnets, {len(corpus.lines):,} lines")
print(f"init  {t1-t0:.2f}s   parse {t2-t1:.2f}s")

154 sonnets, 2,155 lines
init  1.09s   parse 2.21s

The shape of the corpus

Prosodic’s form detection aggregates over every line’s best parse. Asked what meter this is, it answers like a metrist would: binary feet, final-headed (rising), i.e. iambic; five beats and ten syllables per line, invariantly.

mt = corpus.meter_type
{k: mt[k] for k in ('foot', 'head', 'type')}

{'foot': 'binary', 'head': 'final', 'type': 'iambic'}

print('feet per line:     ', corpus.line_scheme['combo'])
print('syllables per line:', corpus.syllable_scheme['combo'])

feet per line:      (5,)
syllables per line: (10,)

How iambic is iambic? Score each line’s best parse (the weighted sum of its constraint violations; 0 = a perfectly alternating pentameter with no mismatch). Most of the corpus sits at 0–2 violations — the meter is real, not a critic’s idealization — with a long tail of rougher lines.

from plotnine import (aes, element_blank, geom_bar, ggplot, labs,
                      theme, theme_minimal)

lines_df = pd.DataFrame({
    'line': [l.txt.strip() for l in corpus.lines],
    'score': [l.best_parse.score for l in corpus.lines],
    'ambig': [len(l.parses.unbounded) for l in corpus.lines],
})

(ggplot(lines_df, aes('factor(score)'))
 + geom_bar(fill='#2a78d6')
 + labs(x='best-parse score (weighted constraint violations)',
        y='lines')
 + theme_minimal()
 + theme(panel_grid_major_x=element_blank(), figure_size=(7, 3.2)))

Figure 1: Distribution of best-parse scores across all 2,155 lines.

A rhyme census

Rhyme detection works from sound, not spelling: each line-final rime is decomposed into nucleus (vowel) and coda feature-edit distances, and pairs are classified as perfect or slant rhyme in that 2-D space (calibrated against Walker’s 1775 rhyming dictionary — slant rhyme turns out to be coda identity with the vowel free, i.e. consonance). Classifying each sonnet’s rhyme-group pattern against a catalog of 39 named schemes:

from collections import Counter

schemes, strict = Counter(), 0
for st in corpus.stanzas:
    sonnet = prosodic.Text(st.txt)
    rs = sonnet.rhyme_scheme
    schemes[rs['name'] if rs else '(no scheme)'] += 1
    strict += bool(sonnet.is_shakespearean_sonnet)

pd.DataFrame(schemes.most_common(), columns=['scheme', 'sonnets'])

	scheme	sonnets
0	Sonnet, Shakespearean	149
1	Sonnet D	2
2	Sonnet, Spenserian	1
3	Alternating D	1
4	Couplet	1

print(f"{strict}/154 pass the strict is_shakespearean_sonnet predicate")
print(f"(14 lines, ~10 syllables, and best-matching scheme = abab cdcd efef gg)")

148/154 pass the strict is_shakespearean_sonnet predicate
(14 lines, ~10 syllables, and best-matching scheme = abab cdcd efef gg)

The 149 that classify as Shakespearean are unsurprising — Shakespeare mostly wrote Shakespearean sonnets. The residue is where it gets interesting. Sonnet 126 (“O thou, my lovely boy”) really isn’t one — the detector reads its twelve lines as aabbccddeeff and files it under Couplet, which is what it is.

Sonnet 106 is the poster child for why the classifier works in two dimensions. Its third quatrain rhymes prophecies/eyes and prefiguring/sing — pairs whose vowels have drifted apart since 1609, so a single phonetic distance can’t hear them (an earlier version of this detector filed 106 under “Sonnet A” for exactly that reason). But their codas are identical, and coda identity with a free vowel is precisely what slant rhyme (consonance) is:

from prosodic.analysis import nums_to_scheme

s106 = prosodic.Text(corpus.stanzas[105].txt)
wf = lambda i: s106.lines[i].wordforms_nopunc[-1]
for i, j in [(8, 10), (9, 11)]:
    a, b = wf(i), wf(j)
    print(f"{a.txt.strip()} / {b.txt.strip()}: {a.rime_type(b)}")
print('detected:', ''.join(nums_to_scheme(s106.rhyme_ids)),
      '->', s106.rhyme_scheme['name'])

prophecies / eyes: slant
prefiguring / sing: slant
detected: abab-c-c-d-dee -> Sonnet, Shakespearean

The classifier still reports what the sounds support — it just now distinguishes which sounds have to match. Disagreements with the print tradition that remain (five sonnets) mark lines where even the consonants have drifted since 1609.

Close reading: Sonnet 18

summary() gives the per-line scansion, rhyme letter, and ambiguity count, plus the estimated schema:

s18 = prosodic.Text(corpus.stanzas[17].txt)
s18.parse()
print(s18.summary())

  #st    #ln  parse        rhyme      #feet    #syll    #parse
-----  -----  -----------  -------  -------  -------  --------
    1      1  -+-+-+-+-+   a              5       10         1
    1      2  -+-+-+-+-    -              4        9         1
    1      3  -+-+-+-+--   a              4       10         2
    1      4  -+-+--++-+   -              4       10         2
    1      5  -+-+-+-+-+   b              5        9         3
    1      6  -+-+-+-+-+   -              5       10         2
    1      7  -+-+-+-+-+   b              5       10         1
    1      8  -+-+-+-+-+   -              5       10         1
    1      9  -+-+-+-+-+   c              5       10         1
    1     10  -+-+-+-+-+-  d              5       11         2
    1     11  --++-+-+-+   c              4       10         2
    1     12  -+-+-+-+-+-  d              5       11         1
    1     13  -+-+-+-+-+   e              5       10         1
    1     14  -+-+--++--   e              3       10         6


estimated schema
----------
meter: Iambic
feet: Complex (5-4-4-4-5-5-3)
syllables: 10
rhyme: Sonnet, Shakespearean (abab cdcd efefgg)

For individual lines, grid_str() renders the best parse as a Hayes-style metrical grid: column height is linguistic prominence, and the w/s row beneath is the metrical template, so a mismatch reads as a tall column over a w (positions with violations are starred). The famous opening line is metrically docile — every strong position is at least as prominent as its neighbors:

print(s18.line1.grid_str())

            *              *         *
            *              *         *
*     * *   *    *    *  * *   *     *
shall I com PARE thee TO a SUM mer's DAY
w     s* w   s    w    s* w s   w     s

Line 3 is not. Rough winds do shake the darling buds of May opens with a stressed monosyllable in the first weak position — the w* under rough:

line3 = s18.lines[2]
print(line3.txt.strip())
print()
print(line3.grid_str())

Rough winds do shake the darling buds of May,

*     *        *         *        *
*     *        *         *        *
*     *     *  *     *   *   *    *    *  *
rough WINDS do SHAKE the DAR ling BUDS of may
w*    s     w  s     w   s   w    s    w  w

Hold that thought: the tolerance of line-initial stress is exactly what the weight-learning experiment below will discover on its own.

Metrical ambiguity

A line’s parses.unbounded are its Pareto-optimal scansions — readings that survive harmonic bounding. Most lines have one or two; a few monosyllable-heavy lines support many more. The most ambiguous lines in the corpus:

lines_df.sort_values('ambig', ascending=False).head(5)[['ambig', 'score', 'line']]

	ambig	score	line
1449	18	4.0	Three April perfumes in three hot Junes burn'd,
1891	17	4.0	And will, thy soul knows, is admitted there;
1321	16	3.0	Making lascivious comments on thy sport,
129	15	3.0	But that thou none lovest is most evident;
451	14	3.0	Gilding pale streams with heavenly alchemy;

The champion — 18 defensible scansions — is from Sonnet 104, a line made almost entirely of stressable monosyllables:

champ = max(corpus.lines, key=lambda l: len(l.parses.unbounded))
print(champ.txt.strip())

def scansion_row(p):
    row = {'meter': p.meter_str, 'score': p.score, 'bounded': p.is_bounded}
    row.update({'*' + k: v for k, v in sorted(p.viold.items()) if v})
    return row

pd.DataFrame(
    [scansion_row(p) for p in champ.parses.unbounded] +
    [scansion_row(p) for p in list(champ.parses.bounded)[:3]]
).fillna(0)

Three April perfumes in three hot Junes burn'd,

	meter	score	bounded	*unres_across	*w_stress	*s_unstress	*unres_within	*w_peak
0	++--+-++-+	4.0	False	3.0	1	0.0	0.0	0.0
1	++--+-+-++	4.0	False	3.0	1	0.0	0.0	0.0
2	-+--+-+-++	4.0	False	2.0	2	0.0	0.0	0.0
3	-+--+-+-+-	4.0	False	1.0	3	0.0	0.0	0.0
4	-+--+-++-+	4.0	False	2.0	2	0.0	0.0	0.0
...	...	...	...	...	...	...	...	...
16	+-+-+-+-+-	5.0	False	0.0	3	1.0	0.0	1.0
17	-++-+-++-+	5.0	False	1.0	2	1.0	1.0	0.0
18	++--+--+-+	5.0	True	3.0	2	0.0	0.0	0.0
19	++--+-+--+	5.0	True	3.0	2	0.0	0.0	0.0
20	-+--+--+-+	5.0	True	2.0	3	0.0	0.0	0.0

21 rows × 8 columns

The bounded rows at the bottom show what bounding removes. Compare the first bounded scansion with the first unbounded one:

ub, bd = champ.parses.unbounded[0], champ.parses.bounded[0]
for tag, p in [('unbounded', ub), ('bounded  ', bd)]:
    viols = {k: v for k, v in p.viold.items() if v}
    print(f"{tag}  {p.meter_str}  score={p.score}  {viols}")

unbounded  ++--+-++-+  score=4.0  {'unres_across': 3, 'w_stress': 1}
bounded    ++--+--+-+  score=5.0  {'unres_across': 3, 'w_stress': 2}

The bounded parse incurs everything the unbounded one does plus an extra w_stress violation: no weighting of the constraints could ever prefer it. That is harmonic bounding — the parser keeps these candidates (the web app’s Line view shows them grayed out) but they can never win.

Learning the meter’s weights

So far every constraint has counted equally. Meter.fit() learns weights from the corpus itself by Maximum-Entropy optimization: make the observed scansion (here, a uniform iambic-pentameter target) as probable as possible against each line’s full candidate set. With zones=3 the violation tensor is split into line-initial, medial, and final thirds, so the model can learn positional leniency:

meter = prosodic.Meter()
meter.fit(corpus, 'wswswswsws', zones=3)

zw = meter.zone_weights
base_names = sorted({n.rsplit('_z', 1)[0] for n in zw})
weights = pd.DataFrame(
    [{'constraint': b,
      **{f'zone {z}': round(zw.get(f'{b}_z{z}', 0.0), 2) for z in (1, 2, 3)}}
     for b in base_names]
).set_index('constraint')
weights

[7.56s] prosodic.parsing.maxent.MaxEntTrainer._build_line_data(): 438/2155 lines had no matching scansion among parser candidates (syllable count mismatch?)

	zone 1	zone 2	zone 3
constraint
foot_size	0.00	0.00	0.00
s_unstress	0.44	0.00	0.00
unres_across	9.81	9.97	10.17
unres_within	8.04	8.75	8.93
w_peak	0.89	1.17	1.14
w_stress	0.00	1.97	1.70

Two things the model discovers match the metrist’s textbook:

• Resolution is the hard currency. The unres_* constraints (splitting a two-syllable position across a word boundary or inside a strong/heavy disyllable) carry the largest weights everywhere — these are the near-inviolable rules of the iambic pentameter template.
• Line-initial inversion is free. w_stress — stress in weak position — costs ~2 in the middle and end of the line but 0 in zone 1. The model has independently learned the license that put Rough winds in line 3 above: a stressed syllable in the opening weak position simply isn’t penalized in Shakespeare’s practice. (Kiparsky’s stricter w_peak, a lexical stress maximum in weak position, stays costly in all zones.)

Finale: phrasal stress

Everything above used lexical stress. With syntax=True, Prosodic also computes sentence-level prominence from a dependency parse — Dozat’s MetricalTree algorithm, ported to spaCy projections (see Phrasal stress; requires pip install "prosodic[syntax]"). Each word gets gradient pstress and tstress values in [0, 1], where tstress = 1.0 is the sentence’s nuclear stress:

s18x = prosodic.Text(corpus.stanzas[17].txt, syntax=True)
s18x.parse()

df = s18x.df
(df[(df.line_num == 1) & (df.form_idx == 0)]
   .drop_duplicates('word_num')[['word_txt', 'pstress', 'tstress']]
   .set_index('word_txt').T)

word_txt	Shall	I	compare	thee	to	a	summer's	day
pstress	0.0	0.333333	0.0	0.333333	0.0	0.0	0.0	1.0
tstress	0.285714	0.0	0.571429	0.0	0.285714	0.142857	0.571429	1.0

The grid consumes these values: phrasal rows extend the columns above the word level, so the nuclear-stress word — day, the point of the question — becomes the tallest column:

print(s18x.line1.grid_str())

                                     *
            *              *         *
            *              *         *
            *              *         *
*     * *   *    *    *  * *   *     *
shall I com PARE thee TO a SUM mer's DAY
w     s* w   s    w    s* w s   w     s

from plotnine import theme

s18x.line1.grid_plot() + theme(figure_size=(8, 3))

Figure 2: The same grid as a figure: line.grid_plot() (plotnine).

Gradient prominence also feeds the parser: the constraints w_stress_p / s_unstress_p and w_stress_t / s_unstress_t penalize phrasally prominent syllables in weak positions (and weak ones in strong) — inert without syntax=True, and most useful for prose rhythm and naturalness ranking rather than fixed-template verse.

Reproducing this page

Everything here runs from a stock install:

pip install prosodic          # + espeak (see Install)

Related reading: Metrical parsing for the theory and the vectorized implementation · Phrasal stress for the MetricalTree lineage · the README notebook for a runnable API tour.