Jeff Ostrowski · October 3, 2023 “Must Hymns Always Rhyme?” • (With Examples) Non-rhyming hymns are certainly a strong temptation since they’re so much easier to compose.