SSH with RSA keys under Raspbian keeps requesting password

Background information

I am trying to connect a Raspberry Pi (Raspbian) to another via SSH. Therefore, I have generated a pair of RSA keys in my "client" Pi and saved its public key in my "server" Pi.

cat ~/.ssh/id_rsa.pub | ssh pi@piserver 'cat >> .ssh/authorized_keys'

Issue

Whenever I try to access my sever via the command below, I am requested to provide a password.

ssh pi@piserver
Enter passphrase for key '/home/pi/.ssh/id_rsa':

Further information

Client ./ssh folder details:

ls -l
total 16
-rw-r--r-- 1 pi pi  396 Feb 28 18:07 authorized_keys
-rw------- 1 pi pi 1766 Feb 28 18:09 id_rsa
-rw-r--r-- 1 pi pi  396 Feb 28 18:09 id_rsa.pub
-rw-r--r-- 1 pi pi  222 Feb 28 13:11 known_hosts

Server ./ssh folder details:

ls -l
total 4
-rw-r--r-- 1 pi pi 1188 Feb 28 18:27 authorized_keys

Question

How can I avoid being asked for the password all the time?