Weird Issue with SSH Keys

[ISSUE RESOLVED] I generated ssh keys (public and private) and stored them in ~/.ssh/myServer/. I added the public key to the server's ~/.ssh/authorized_keys file and everything works great! In my effort to try and understand how the keys work I deleted them and was surprised to find that I was still able to connect to the server. Why is it that I am still able to connect without a password? (The client is a Mac if that makes any difference.)

[SOLUTION] Turns out OSX has a ssh-admin which stores the private keys until you restart. Even though I had removed the private key (id_rsa), ssh-admin still had it stored in memory and was using it to allow me to authenticate. Once I rebooted it was no longer in memory and I was unable to connect. I bit tricky...