Because something else is already there waiting to gobble them up: dust mites. And the longer you wait between washes, the more food these critters will have and the more they'll procreate and ...