The regulator's new ranking system for superannuation funds has shaken up the industry, but investors have been warned not to over-react to some of the early indications.

The Australian Prudential Regulation Authority made headlines last month when it launched the MySuper Product Heatmap.

According to APRA, the heatmap provides “credible, clear and comparable insights” into the outcomes provided by MySuper products in a number of key areas.

"The heatmap is expected to drive improvements in outcomes for members by holding licensees publicly accountable for their performance, and in particular highlighting areas of underperformance,” APRA said on the heatmap’s 10 December launch.

It is this last point about underperformance that has drawn criticism. Concentrating on this aspect of the heatmap provides a skewed assessment, according to some commentators.

Financial Services Council CEO Sally Loane notes that the heatmap doesn't tell super members how their fund has performed over their lifetime, or indicate how it is managing the risks of a market downturn.

Similarly, Martin Fahy, head of the Association of Superannuation Funds of Australia, says: "Achieving sound investment performance and broader member outcomes is a long-term journey, it’s not measured in terms of years, it’s measured in terms of decades.”

Graham Hand, editorial director of Morningstar and Firstlinks says the fault lies with some media commentators who are singling out funds with lower performance scores, rather than with the heatmap system itself.

"If you just look at the numbers, you know 5 per cent versus 8 per cent, if you're in the lower performing fund you might think you're not doing very well," Hand says. "But the heatmap is not actually measuring that."

The heatmap measures the performance, fees and costs and sustainability of member outcomes of the 416 MySuper funds. Results are depicted using a graduating colour scheme, where white indicates the most positive assessment, while dark red is the most negative.

Knee-jerk reactions could hurt individuals

Poor performance scores could ultimately have a negative outcome for individual members, but not for the reasons you might think.

Many super funds that are more conservatively positioned, with lower allocations to global and local stocks that have delivered high returns in the last couple of years, have lower performance scores.

The danger lies in how trustees respond to what they may see as criticism from the regulator and the popular press.

Hand says some trustees might say “we look bad against some of our more aggressive competitors, we want to have the same asset allocation as them”.

"But if the market has a difficult couple of years, then that fund would have made the wrong change at precisely the wrong moment, and they didn't have the run-up [of solid returns in the previous 12 months or more.

"If they decided they couldn't tolerate the underperformance any more, they become aggressive and then the market falls, that's the worst of both worlds."

This emphasises the importance of taking both a long-term view of your portfolio instead of concentrating solely on the new heatmap's performance criterion.

The APRA league table currently only scores fund performance over three and five years. While this is longer than the one-year figures often cited in the press, it's less than the 7- or 10-year timeframes used by most of the people who manage super fund portfolios.

The framework currently only focuses on one part of a market cycle – an equity market upturn – rather than a full cycle. This is something APRA intends to address by extending the timeframe.

The fees component of the heatmap is also something individuals should examine, but within reasonable limits.

APRA's heatmap looks poorly on funds with higher fees relative to their peers. But high fees aren't a bad thing if the manager delivers superior returns. This is a point emphasised by investment consultant Phil Graham in a recent article for Firstlinks.

"The total fees outcome needs to be assessed alongside the net investment return measures. Indeed, some of the results show funds with the highest fees have delivered the best results after fees," Graham says.

Morningstar's Hand also believes there is an over-emphasis on percentage fees versus the actual dollar value.

"If you think about a fee of 2 per cent on $10,000, that's only $200 a year or about $4 a week," he says.

"You might think 2 per cent sounds like a high fee. But for that, you're getting professional funds management and you get a call centre you can ring up any time, and you get good reporting.

“But when you consider a higher super balance of $200,000, the same fee level amounts to an outlay of $4000 a year – a considerable cost.”

Hand suggests investors consider both the amount they have invested and the fee, rather than simply dismissing fees as either too high or too low on face value.