High speed bits are intended and optimized (point shape, rake angle and material) for drilling hard, tough metals like low carbon steel. They will certainly make a hole in plaster dry wall board and wood and will not be damaged if you hit a nail. But they won't drill as clean and smooth a hole as a bit sharpened in a pattern better suited for these soft materials. Their tip will tend to wander around a bit as you start the hole and the hole will come out a bit oversized.
You may be able to find (and for less money) brad point drills (they have a little sharp point to stabilize the center of the hole as you start, and cut a circle out, first, then clear the rest of the hole)
http://www.amazon.com/Steelex-Plus-D2340-4-Inch-Point/dp/B0000DD1YN/ref=sr_1_7?
or pilot point bits (with a smaller diameter drill point at thew tip)
http://www.amazon.com/Dewalt-DW1263-14-Piece-Cobalt-Pilot/dp/B00004RGVZ/ref=sr_1_1?
are made for softer, and fibrous materials. These are often not made of high speed steel, in which case might get dull if you hit a nail.
They may be called, simply, wood bits.
--
Regards,
John Popelish