Can I limit CPU usage? - Already tried nice value

Hello,

I have a dedicated server running Centos6 x64 and I use it to host minecraft servers. However, one person is always using more CPU than I'd like (peaks of up to 500%) which is causing terrible performance issues for other servers.

Is there any way to limit CPU usage for specific users? (And I mean users, not a single PID) I already tried a nice value, but that didn't really work, I just want to soft limit the CPU usage at about 80-95% so he can peak over at times but not more than x seconds.

Is there any way to do this at all? Preferably without requiring a server restart.