Virtue, according to the utilitarian doctrine, is not naturally and originally part of the end, but it is capable of becoming so; and in those who love it disinterestedly it has become so, and is desired and cherished, not as a means to happiness, but as a part of their happiness. [Utilitarianism, Chapter 4]
The implication is that pleasure/happiness is more primal, more contingently precedent than any such refinement as a disinterested taste for virtue. Underlying this is the thought of our evolutionary connection to beings only capable of less abstract desires: We came from them. They can only do that. Therefore, we started out only being able to do that. Most of us, most of the time still barely rise above that. And if we occasionally do, we do so only by an extension, a stretching of what pleasure or happiness means. But as living beings we can only ultimately desire the pursuit of happiness as an end. Mill wants to deny a qualitative change in the nature of what human beings can wish for. The transcendentalists, by contrast, do detect a qualitative change.