In my experience the difference between a light, early 180 (have flown 55, 56, 57, 59 & 64) and a later 185 (76) is pretty vast from a pilot's perspective. That is - they are completely different machines - as many have already mentioned. They simply don't look, feel or fly the same inside the cockpit. The 185 feels like a much larger aircraft with heavier controls and a more ponderous feel - much like the difference between a 182 and 206. However, unlike the 182 and 206, they are in fact essentially the same aircraft with slightly different features. Personally, I attribute the differences to weight, years and engines. (Early 180s generally seem to be in the 16-1700lb range assuming they are not decked out to be a 185: the 185 I work is 2100lbs empty on wheels.) I would imagine as the years and equipment lists get closer, so does the feel in the cockpit. Many of the controls change as well as the years go by; such as updated breakers and switches, fuel injection, and a more instrument-friendly panel. Many of the older 180's have gauges, switches and controls seemingly randomly scattered across the mostly empty dash - kinda feels like an old Ford. But what a honey to fly....
If you need range and load hauling ability the 185 is hands down better. With Flint Tip Tanks the 2100lb beast still has a 1500lb useful, plus over 8hrs usable fuel and full IFR capability with most any gadget a pilot could wish for. Sure it could be lightened up some, but the empty weight includes a FULL arctic survival kit for 4 folks and all the instrumentation you could ever need. It comes in handy too sometimes... All-in-all a real workhorse for a specific mission. Sure a 206 will do the same job or better most of the time - but not when wheel-skis are required.
On the other hand, a good 180 can be a load hauling, traveling machine extraordinaire for as little as half the initial price. And for short, rough airstrips hauling moderately heavy loads - I'll take a light 180 with a large motor. Don't have any experience with early 185's - they may be the same. Nimble feeling and responsive when you need it and still cruises fast. Burns the same amount of fuel or more than a 185 without the range, but lightweight is lightweight on the same wing. Granted a similar horsepower, a lighter plane will have the advantage for hauling a heavy load out of a tight spot as the total weight will be less.
Excellent aircraft all - they just vary by degrees and as such can be tailored to a wide range of missions.