I'll have to give the instant ice thing a try.  Maybe it is the acid spoiling the meat, over the dark meat that I find disgusting?  I'm guessing that it affects different species at different rates.  I'm wondering if the temp of water the fish is in, also affects the acid or taste.  Most of my wading is in cold water.  When boating, in warmer months, I usually toss directly on ice.  The wading fish usually taste better... 
Last point is also panfish over trout, since my trout fishing is maybe 2-3 trips per year.