Did anyone get Borgbackup excludes right the first time?

I want to backup my home directory and leave out heavy directories like my git repos, videos, tmp dir, VMs and such out so I can back them up differently and keep my home directory backup small enough.

I've tried a large range of --exclude /path/to/dir1 variable in my script and that didn't work.

Then I realized borg has a --exclude-from /path/to/excludes switch which I've tried. It also ignores it seems absolute paths.

I've then tried using the sh:(for shell expansion and such) prefix for absolute paths and that didn't work.

Then last time, I've just used relative directory and file names compared to the backup point(eg /home/user) and only those worked in the backup script.