Need to restrict potential access to teamviewer from outside world

My office firewall is running an old version of OpenBSD and pf

Is it conveniently possible to restrict teamviewer incoming connections to a certain PC? If I know the public IP of the user that will be attempting to connect can I allow incoming connections ONLY from that PC?

Boss does not want to bother with a VPN solution for this situation as the employee will only need remote access on a couple days.

I'd prefer not to allow open access from the outside world if at all possible.

I understand that teamviewer is in theory relatively secure but I would still rather not risk it at all if I could avoid it.

Also, this is for a completely non-technical user.