sudo rights to chown but limited to one folder and subfolder?

I need to do backups of Servers that don't have enough free space on them to store the backup so I'm rsync'ng all the files to the Backup Server and then tar.gz'ng them.

But since I want to keep ownership rights I need to give the backup user sudo {without password} access to chown.

Is there any way to limit it just to one folder and any subfolder off it?