Borg or the like with ‘hardcoded’ plaintext/regularly full-disk-encrypted key is acceptable. Someone that has your unencrypted private key sitting on your server has almost certainly already obtained access to the entire set of data you’re backing up, with the backup key itself only meaningfully guarding access to older backups.
The more important thing is to securely keep extra copies in case the server fails. I keep mine in a group in my password manager, one per repo.
Love the diagonal belts & power poles.