Next to the Undelivered Message icon (red triangle with an exclamation point inside) is an ‘X‘. This X allows you to delete unseen, undelivered messages.
If you click on this ‘X’, the Confirmation box will pop up. As before, you can clear all unseen undelivered messages. Now you can also choose to unsubscribe contacts if their last 2 messages were undelivered.