The nominal charging current is 1/10 of capacity, so 80mA. Your panels are only slightly higher. NiMH can be charged faster so long as they do not get too hot (45°C is a typical limit).
At the 0.1C rate it takes 15 hours to charge from "empty". At 167mA it would be half that time, but this is a "fast charge" that needs better management.
The nominal cell voltage is 1.2V, but more like 1.42V when fully charged. That suggests the solar panels must deliver over 5.7V at the battery terminals to fully charge the battery.
It seems the 9V panel is a better match, able to fully charge the battery, considering that the panels get hot in the sun, and voltages reduce due to the voltage temperature coefficient of the solar cells (-2.2mV per cell per degree C typically). In the full sun they can get over 50°C, so a 25°increase. A 9V panel is likely 16 cells, so the drop would be to around 8V. The 6V panel is too low to fully charge the battery. Call it marginal.
Charging requires a source that is current limited, but this is no issue here because the panels are inherently current limited. The voltage of the panel is just pulled down to the battery voltage, and the current is according to illumination.
There needs to be a way to cut the charge when the battery is fully charged. Voltage is not a very good indicator with NiMH batteries. They can be charged according to time if starting from a known charge. This is "empty" which is around 1.0V. If you charge for 15h and then cut the charge, it may be overcharged if it was not empty, but the NiMH is designed to handle this if it is for 15h, and at the rate of 0.1C (or less).
One way to detect full charge is called "delta V". There is a bump in the voltage at full charge. This means not just monitoring the voltage, but remembering the recent voltage, so the bump is detected by the rate of change in voltage. I needs appropriate hardware, usually a microprocessor. Temperature is a better indicator. It increases when the charge is full. This is a delta T measurement really, but often it is a big enough change that it can be just sensed as a temperature increase.
The solar panel only generates 100mA when:
o In full sun.
o At a high angle in the sky in both x and y planes (>45 degrees maybe).
o Clear sky.
o At right angles to the sun's rays in X and Y planes.
The average output over a day can be stated as "equivalent full sun hours". This link has excellent explanations and detail.
http://www.mpoweruk.com/solar_power.htm
Note that insolation is measured in a slightly different way to a solar panel, but you can ignore that for this situation and call it equivalent full hours.
The battery charge = average current x time, in ampere hours (Ah or mAh). The total Ah produced per day is not going to charge the battery in one day. We can see from the above it might take 4 days or more, averaging 4h a day.
This is an argument in favor of the higher current output panel, except that its voltage will not charge the battery fully. It will need charge management though (temperature monitoring of the cells) because the current produced can exceed the ability of the cells to look after themselves. The ideal solution is a 9V or higher panel voltage, with a larger current rating (bigger area) and a properly matched charge control for NiMH batteries. The last item might be difficult to find. The 9V panel you have is right for a simple charge system, except that it takes a few days to charge, which is probably fine. The Ah usage (by the load) should be less than the panel provides, so charging can catch up.