For Signal/Molly, it’s less that the notification is encrypted as I understand it. It’s more the notification content is just “Hey! Stuff happened” for Signal. The app then reaches out directly to the Signal servers to see what’s new. So the message content is never sent via the push notification service (UnifiedPush or Google’s service).
I’m self hosting both too. MollySocket’s docs are pretty clear that it never gets an encryption key for your account, so it can’t read your messages. It only gets/forwards alerts that something happened on your account AFAIK. So I’m not sure what data it has that’s worth encrypting.