Jack Viertel, The Secret Life of the American Musical: How Broadway Shows Are Built (Farrar, Straus and Giroux, 2016)
Underneath the author's "aw, shucks" manner, this is a fiercely intelligent book on the construction of the classic American musical. Viertel is less interested in how the musical is actually made by the creators than on how it works for the audience. The characters and story have to involve the audience adequately, or else the musical will fail. Fortunately he's more interested in showing examples that work well than those that don't, but he does give enough of the latter to emphasize the point. (For instance, Camelot introduces Arthur and Guinevere each with a song about how they don't want to be there, and this makes the audience not want to be there either, even though they're good songs. And indeed, Camelot was not a stage success, and Viertel says this is why.)
Viertel's larger thesis is that the classic musical was built to a formula - not mechanically, which will fail, but by tried and true principles that work when used with imagination and flexibility - and that the fall of the form (he says the last classic was A Chorus Line, 1975) came when this was forgotten. He says the problem is that the classic musical was a hopeful, optimistic form (an odd thing to say when his prime examples of greatness are Carousel and Gypsy), and that the cynicism that arose in the 1970s made this no longer possible. (So far he sounds like Peter Thiel's theory of SF, that it was all cheerful, optimistic stuff before the 70s, suggesting that Thiel has never read any of it.) Some writers, like Sondheim, had grown up in the old regime and were intelligent enough to adapt it to the new circumstances and do great work, but many new ones didn't and failed through fumbling around. (And that sounds like a critique of post-Tolkien fantasists.) Yet he praises some newer work, notably The Book of Mormon and Hamilton.
The book is made out of chapters considering each type and placement of song a musical might have. Viertel doesn't use much of the old terminology I've heard of: he folds the "'I Am' song" into the "'I Want' song," and defines the "Eleven o'clock number" loosely to mean any searing event near the end of the show, even a non-musical one.
I'll go through Viertel's analysis of Act One to show how this works (Act Two tends to be messier and more conditional). He gives lots of examples; I'm citing mostly just a few with which I'm familiar.
1. Opening Number. This has the vital function of setting the tone and telling the audience what kind of show this is and what it's about. What kind of show is A Funny Thing Happened on the Way to the Forum? "Comedy Tonight." What is Fiddler on the Roof about? "Tradition."
2. The "I Want" Song. The canonical example is "All I want is a room somewhere / Far away from the cold night air / With one enormous chair," but Viertel also defines Higgins' opening number as "I Want" (I want the English to learn to speak properly) though I'd define it differently as "I Am" (I am obsessed with language and social class). "I Want" songs can be subtle, but most of them aren't: "I wanna be a producer."
3. Conditional Love Songs. Best explained by citing the canonical example, "If I Loved You" from Carousel. The characters claim they're not in love yet, they're just going to talk as if they were. There are also sub-types: the aftermath song, in which a character alone tries to make sense of a perplexing romantic encounter ("On the Steps of the Palace," Into the Woods), or the buddy song ("We Can Do It," The Producers).
4. The Noise. To provide a little break and an energy boost. "Put on Your Sunday Clothes," Hello Dolly; "To Life," Fiddler on the Roof.
At this point, the story can take a number of turns. But there will usually be songs for one or more of the following:
5a. Second Couples. Ado Annie and Will in Oklahoma! Cable and Liat in South Pacific. Tuptim and Lun Tha in The King and I, who were introduced to have the big romantic ballads, inappropriate for the leads, who weren't that kind of singers anyway.
5b. Villains. Jud in Oklahoma! Judge Turpin in Sweeney Todd. The dentist in Little Shop of Horrors.
5c. Multiplots. "No Life" in Sunday in the Park with George, which prepares the story for the shift in perspective in the second act.
Then the plot regroups with:
6. Star Turns. A big mid-act showcase for the star. Viertel finds the most interesting example to be "Adelaide's Lament" in Guys and Dolls, because the original performer wasn't a big star but the song showcases her anyway.
7. Tent Poles. Another high-energy number, this one more usually focused on a crux in the plot. The Havana sequence in Guys and Dolls. "Tevye's Dream" in Fiddler.
8. Curtain, Act 1. The presentation, or sometimes the anticipation, of a crisis. "Everything's Coming Up Roses" in Gypsy. "Soliloquy" in Carousel. Both of these are also big solo turns, but they're more dramatic than they are showcases.
Get the idea? Viertel presents all this very well.