Wednesday, August 8, 2007

The GB Response

Thank you for contacting GarageBand Support. All reviews weigh in the
same and the algorithm is not something I can pretend to understand.
Here is a bit of information to help you understand the process... this
was written by a manager...

Prior to the genesis of the star ranking system, there was no good way
for the user to gauge how well a song was doing. They could tell that
their song was at X position in the charts, but there was no way to
really tell how a song was doing against its peers because all of the
songs are shifting positions.

We responded to overwhelming demand from our user base by creating the
star system as an early predictor of how the song is doing against the
galaxy of its peers. As a predictive indicator, the star system is not
highly accurate in the early stages and becomes more accurate as the
song approaches the end of its reviews.

One inherent problem right now is that both the chart positions and the
star rankings are calculated not only by the song's reviews, but by ALL
reviews of the song AND ITS PEERS. This causes understandable
confusion
among users.

For example, if I enter a new song, it will go through a period of
heavy
rotation and climb the chart. At some point, the review frequency will
decrease but OTHER songs are getting reviews. I may see a single
review
or even no review at all and on the same day see that my chart position
and/or my star rating will move quite a bit. It would be natural for
me
to assume that this condition was caused by the single reviewer when in
fact it is because there is a whole lot of other data being
considered--namely, the data of all the other songs in that genre that
are receiving reviews.

In truth, a single review can't substantially move a song's position
because we are using quantitative rather than qualitative analysis. I
do understand, however, that this can be the perception and that this
perception is very important.

We are planning to address the way our star system calculates the
ranking. While there is no way to 'fix' the inherent inaccuracy of a
narrow sample, there are steps we can take to offset the star ranking
to
reflect the confidence in the sample accuracy.

At this point, we have not fully discussed how we would implement this
but it is something of which we are aware and intend to fix.

Thank you,
Jennifer
GarageBand.com

No comments: