The inexpensive analog moving-needle meters you can get at hardware stores and Radio Shack measure average current, and thus average voltage. They are calibrated to read as RMS voltage, but that calibration is true only in a very small frequency band, 50 Hz to 60 Hz. You cannot measure the voltage coming out of an audio amplifier with them.
Digital meters can be designed to display anything.
You have to read the specification sheets to know for sure.
http://www.myflukestore.com/p1228/fluke_179.php
For example, the Fluke 170-series of digital meters have this specification:
Voltage frequency is specified from 2 Hz to 100 kHz.
Current Frequency is specified from 2 Hz to 30 kHz.
Inexpensive digital meters only work with 50 Hz to 60 Hz voltages and currents.
Most electrician and industrial digital multimenters are designed for the 50 Hz to 60 Hz range.
Take a look at the Fluke website and explore looking for the specifications and data sheets.
For example, on this webpage http://www.myflukestore.com/p3716/fluke_117.php
look for the tab "Documents" and you can download a lot of very informative PDF files.