Ibn Mas`ud (May Allah be pleased with him) said:
The Messenger of Allah(Sallallahu Alaihi Wasallam) said,
“When three of you are together,
two of you must not converse privately
ignoring the third till the number increases,
lest the third should be grieved.”
[Al-Bukhari and Muslim].

Commentary: 

This Hadith tells us that holding private counsel has been prohibited
for the reason that it hurts the feelings of the Muslim who is ignored; and to hurt
the feelings of a Muslim is a great sin indeed.

ALLAH, Subhanahu wa ta’ala, says:
“And those who annoy believing men and women undeservedly, bear on themselves the
crime of slander and plain sin.” (33:58)

Whispering of the two is permissible when all the three mix up in a crowd.

Then the two can speak to each other in confidence.