A better definition of an (ideal) voltage source is: a device which maintains always the same voltage across its terminals, no matter how high may be the current it has to supply. Likewise, an ideal current source is that which supplies always the same current to anything connected to its terminals, irrespective of how high has to be the voltage developed to ensure this is so.
Real sources can only approximate this behavior ; for example, an actual voltage source (a storage battery, say) supplies a constant voltage up to certain point. If current increases beyond this point, terminal voltage across the source will drop somewhat. A real voltage source has always some internal impedance. As you increase current, at some value the effect of this internal impedance starts to be noticeable.
Analogously, as you increase the value of the impedance connected to a real current source, a point will be reached at which the source will not be able to develop the high voltage required across its terminals in order to keep current constant.
The above discussion should have made clear that a voltage source can --and does-- deliver current; in addition, it does so keeping voltage across its terminals reasonably constant.
A current source tries to keep current through itself reasonably constant, even if this implies having to adjust terminal voltage over a very wide range of values, according to the load connected to it.
Only an infinitely high (in ohms) load will draw no current from a voltage source, i.e., an open-circuit. Otherwise, a current will flow; its intensity ultimately depends on two factors: voltage put out by the source, --assumed constant-- and load impedance. That is, if you connect a 12 V source to a 120 Ω resistor, a current equal to 12/120 = 0.1 A will flow.
In contrast, only a short-circuit can prevent a current source to develop some voltage across its terminals. Otherwise, there will be a terminal voltage which ultimately depends --again-- on two factors: current, which is already known, and load impedance. The same 120 Ω resistor, connected to a 1 A current source, will develop a terminal voltage of 120 V.
Hope this will help.
14/2/2007 update.
Sure. That's what I've trying to say, all the time. In fact, a voltage source can be compared to a water pump, or rather, a constant-pressure water pump; if the pressure is there, usually water flows. Unless stopped by a (closed) valve, which can be compared to a switch. Or a totally clogged pipe, which is like an open circuit, or an infinitely high resistance (high in ohms, not height).