Police are investigating after paint was thrown at a mosque and a rock hurled through a window while worshippers were inside.