The most common usage being that last: herbaceous plants don't have woody stems (or to use technical jargon, no lignified tissues), and perennial herbs die completely off each winter to regrow from the root. Grasses and forbs are both herbaceous.
This week on my main journal I'm running a theme of words first used by 17th century writer Sir Thomas Browne, who wrote with a notably Latinate style -- and somewhat predictably, brought a lot of words from Latin into English, many of them commonly used today. And, yeah, this is one of them: adopted by Browne in 1640 from Latin herbāceus, grassy/like grass, from herba, grass/herbs.
We hacked our way though the brush until it thinned to an herbaceous undergrowth.