# TC rankings exposed as BS



## RogerWaters (Feb 13, 2017)

In my last post on the unreliabiltiy of TC lists (I actually meant lists, not polls, as I incorrectly titled the thread), I was challenged to justify why they are basically BS.

I have conducted 5 polls which test the rankings of various TC lists:

1. The TC 150 Most Recommended Symphonies with this poll pitting Beethoven's 4th to Sibelius's 4th
2. The TC Top 100 Recommended Choral Works with this poll pitting Bach's St. John Passion with Haydn's The Creation.
3. The TC Top 50 Recommended Piano Chamber Works with this poll pitting Brahms's Piano Trio No. 1 with Mendelssohn's Piano trio No. 1
4. The TC Top 100 Recommended Chamber Duo Works with this poll pitting Brahms's Violin Sonata No. 3 with Ives's Violin Sonata No. 3, and
5. The TC Top 200 Recommended Solo Keyboard Works with this poll pitting Beethoven's Piano Sonata No. 28 with Schoenberg's Drei Klavierstück Op. 11.

Every single one, at the moment, inverts the findings of the TC lists.

What does this 'prove'? I'm not one to present findings as vindicating a stronger thesis than they actually do. Thus, I don't think this proves TC lists are completely pointless. I have discovered much great music using them. However, I think what this shows is that the rankings are completely erroneous. If people don't take them seriously in the first place, then fine, you've nothing to see here. However, if you do, then arguably you shouldn't.

Explaining the results: My guess is that the arbitrary rankings of TC lists is due to the following phenomena: I list my top 10 symphonies, and because I like Beethoven's 9th and a bunch of others over his 4th, I place a vote for Beethoven's 4th as being somewhere at the bottom of the list. However, this doesn't mean I have had the chance to pass my judgement upon, say, Sibelius's 4th, which a bunch of other people have placed near the tops of their lists - where they haven't included any beethoven at all. So beethoven's 4th gets a judgement which places it towards the bottom, even though I might think it's better than Sibelius's 4th.

Basically these kinds of rankings assume everyone has consciously voted on the specific ranking of each work, which is far from the truth. Such a methodology leads to really false results. I'm sure someone with a background in statistics can explain the problem much better than I can.


----------



## Ethereality (Apr 6, 2019)

Polls may correlate to popularity: they will tell you 'how many people who don't care much about the question' will vote for option A or B. For instance, in a poll you can have 100 people vote for A, and that means they're giving it a +1 with no indication of how _much_ they like it or if they've even listened to the other one. In a survey on the other hand, you may often have more people put B on their top favorites instead, and this would indicate B has a bunch of _+5s_ not a bunch of +1s. Surveys will tell you how _much_ people like something.

In the new *Talk Classical's Favorites Works Project*, you can submit an unranked list of your favorite works now. People have been sending in their favorite 10 or 20 works of all time, and even 100. No ranking required. The average charts just update with new entries.


----------



## UniversalTuringMachine (Jul 4, 2020)

Before exposing "X" as BS, it's best to take an elementary statistics class first.


----------



## Trout (Apr 11, 2011)

Did it not occur to you that nearly all of those lists were conducted several years ago and thus had an entirely different group of people voting than the people polled in the last week?


----------



## RogerWaters (Feb 13, 2017)

Trout said:


> Did it not occur to you that nearly all of those lists were conducted several years ago and thus had an entirely different group of people voting than the people polled in the last week?


I don't think it would have changed things much, given the works I chose. Of course, I might be wrong.


----------



## KenOC (Mar 7, 2011)

What these lists prove is that too many TC denizens are driveling cretinous idiots with the musical taste of diseased marmots. Comparing Beethoven’s and Sibelius’s 4th Symphonies is a case in point: It will be immediately obvious to any normally-intelligent and mentally-fit person which is superior. But the brain-dead inbred imbeciles in this forum, many with family trees that look like telephone poles, will get it wrong every time.

So yes, we have some problems with lists (and with polls for that matter). But nothing is to be done, so it’s probably best just to smile and act as if all the others are fully as brilliant as are we ourselves.


----------



## starthrower (Dec 11, 2010)

I hardly ever look at them. It's just not my preferred way of discovering music.


----------



## SanAntone (May 10, 2020)

starthrower said:


> I hardly ever look at them. It's just not my preferred way of discovering music.


I enjoy the games but don't follow or read the lists. But if others find them worthwhile, it is good that they exist.


----------



## RogerWaters (Feb 13, 2017)

Ethereality said:


> Polls are inaccurate, surveys are accurate. in a poll you can have 100 people vote for A, and that means random people are giving it a +1 with no indication of how much they really like it. In a survey, you can have 50 people put B on their favorites list, and really only 25 people will put A, and that would mean B wins by twice the favor.


Hang on, why are people replying to the polls just 'random' while the people replying to the surveys are 'legitimate'?

BTW, i'm not holding that polls are accurate and surveys not. I would think both have significant problems. However, I'm not seeing why surveys are supposedly accurate and polls are not, simply because surveys contain information about 'how much' a piece X is liked (less than Y) and how much piece Y is liked (more than X), when a different test shows that most pople like Y more than X!

Maybe i'm not understanding you...


----------



## Trout (Apr 11, 2011)

RogerWaters said:


> I don't think it would have changed things much, given the works I chose. Of course, I might be wrong.


TC is not a monolith. It is comprised of individuals who come and go, thus changing the highly-varying complexion of tastes and preferences. Take the variety in the three opera lists conducted over the last 10 years, for example:

- 2011 (This thread was in the opera subforum, if I recall correctly, which was almost hermetically comprised of members much more passionate about opera than CM at large. Quite traditional overall with some eccentric favorites.)
- 2015 (This was during the peak of listeners passionate about modern/contemporary music as you can see with the fairly high placing of works by Berg, Glass, Adams, Saariaho, etc. Many of those members have since sadly left.)
- 2020 (This seems to reflect a reversion back to very traditional tastes, but even still it's hard to call it completely representative as there weren't that many participants. I checked and the last couple rounds had only 7 members left participating.)

You can undoubtedly find hundreds of contradictions throughout them, but to even think of them as "contradictions" misses the point entirely.


----------



## Ethereality (Apr 6, 2019)

RogerWaters said:


> Hang on, why are people replying to the polls just 'random' while the people replying to the surveys are 'legitimate'?
> 
> BTW, i'm not holding that polls are accurate and surveys not. I would think both have significant problems. However, I'm not seeing why surveys are supposedly accurate and polls are not, simply because surveys contain information about 'how much' a piece X is liked (less than Y) and how much piece Y is liked (more than X), when a different test shows that most pople like Y more than X!
> 
> Maybe i'm not understanding you...


That's a great point, I don't really have an answer: I know people tend to put a lot of thought and dedication into their favorites lists, from responses I've seen. See the bit I added to the end of my last post, that's very significant for surveys.

If you use the correct ranking algorithm, then ranked surveys may simply be much better, more precise, and easier to do.


----------



## hammeredklavier (Feb 18, 2018)

RogerWaters said:


> I don't think it would have changed things much


See how these polls started at different times, produced different results:

https://www.talkclassical.com/class...ls/poll-28-your-favorite-classical-music.html (Jan-09-2007)
Baroque 15.96%
Classical 14.89%
Romantic 47.87%
Modern 21.28%
What period/era you think is ultimately the best? (Nov-24-2019)
Renaissance 4.17%
Baroque 12.50%
Classical 12.50%
Romantic 32.29%
20th Century 34.38%
21st Century 4.17%


----------



## Ethereality (Apr 6, 2019)

I should also add, I've used this website https://rankingengine.pubmeeple.com/ to rank my own favorites lists before, and this essentially polls 1 against another for you until the engine lists all them in order of who won. I've found it to be very accurate and helpful, which seems to be a favor towards polls.


----------



## hammeredklavier (Feb 18, 2018)

KenOC said:


> Comparing Beethoven's and Sibelius's 4th Symphonies is a case in point: It will be immediately obvious to any normally-intelligent and mentally-fit person which is superior.


So which is superior? I'm really curious to know.


----------



## UniversalTuringMachine (Jul 4, 2020)

1. A vs B pairwise voting leads to Condorcet paradox

2. First poll and the last poll are not statistically significant, provided that the samples are random, which are not

3. For each TC list of 50-200 works you have only shown at most a single inversion, which says almost nothing statistically

4. As Trout has pointed out, your poll is not nearly as representative as the TC lists, take a look at the views

5. Poll has greater self-selection bias because participants cares more about A vs B than non participants.

6. The title of the post is sensational and insulting to people who contributed to the TC lists


----------



## ORigel (May 7, 2020)

1. A different group of people voted on the lists than your polls.

2. People's preferences can flunctuate. I was in aBrahms phase a month ago; now I'm in a Handel phase.


----------



## KenOC (Mar 7, 2011)

hammeredklavier said:


> So which is superior? I'm really curious to know.


I believe you know already. Whether you're right or wrong is another question!


----------



## Guest (Jul 22, 2020)

I thought the sole values of the polls, games, or lists is that they are amusing. Apparently that escapes some people.


----------



## Bulldog (Nov 21, 2013)

Baron Scarpia said:


> I thought the solve values of the polls, games, or lists is that they are amusing. Apparently that escapes some people.


It sure does, but we have some members who attach a weight to these threads that doesn't exist. They are snapshots in time of the musical preferences of the participants.


----------



## BachIsBest (Feb 17, 2018)

Baron Scarpia said:


> I thought the sole values of the polls, games, or lists is that they are amusing. Apparently that escapes some people.


It's also a nice way to determine what the 'warhorses' are when you're first listening to classical music. It's such a broad area that's very difficult to determine where to dive in deeper and I know when I first started listening a couple of years ago such lists really helped me find new works. It's funny to me now that I would 'find Bramhs's 4th', for example, a new work, but if somebody (like me a couple of years ago) doesn't know much beyond The Four Seasons, Tocatta and Fugue in D minor, and Beethoven's 9th, this can be helpful.


----------



## Guest (Jul 22, 2020)

RogerWaters said:


> In my last post on the *unreliabiltiy *of TC lists (I actually meant lists, not polls, as I incorrectly titled the thread), I was challenged to justify why they are basically *BS*.
> 
> I have conducted 5 polls which test the rankings of various TC lists:
> 
> ...


You've sprayed negatives around without reducing your criticism to a single bone of contention. That is, IMO, unwise.

First, as others have said, any poll or survey (I'll just say 'poll' from now on) is only a snapshot of the opinions of the people who voted at the time the question was posed. To draw any conclusion from the results as 'true or false' is absurd. The only way such a poll could be 'false' is if the person compiling the results made a mistake in the mathematical compilation.

Second, regardless of whether the poll is deemed accurate, it can still be useful, depending on the purpose to which it is put - and who is putting it to the use they wish to put it to.

Third, as far as I can see, you take no account of who is voting and why. I might be attracted to a one v one poll (as I was in the case of LvB 4th v JS 4th) because I like the works concerned, but have no interest in ranking my top 5/10/100 works/composers. Assuming I'm not the only cussed sod here, you'll find you're polling different audiences depending on the question asked.

Last, whilst repeating polls over time can show _changes _in people polled and changes in tastes, it also shows _consistencies_ which, IMO, are sufficiently frequent to confirm that Beethoven, Mozart and Bach are the three most highly regarded composers of CM among the general CM-listening public (not necessarily a more highly specialised CM listening group). It also shows consistencies in the same characters generally appear in the top 10 and the top 20.

Anyone who thinks that you can poll with any degree of exactitude that Sibelius and Massenet must be in 11th and 47th place respectively has an over-inflated idea of the ability of any poll to make such nice distinctions in matters of taste.


----------



## mmsbls (Mar 6, 2011)

These lists were part of a series of projects done years ago using a distinctly different methodology than more recent lists. The titles of these projects all include "The TC Top ... Recommended ... Works." They are not intended to be definitive lists of the most liked or greatest works. The goal was to produce a list of works in various genres that people could use to find excellent music they wanted to sample (i.e. recommendations).

The projects did not involve members submitting a single list of works. The actual methodology was more complex involving members submitting partial lists in nomination rounds and then voting on the most nominated works in voting rounds creating 10 ordered works per round. For more detail you could look at this thread.



RogerWaters said:


> What does this 'prove'? I'm not one to present findings as vindicating a stronger thesis than they actually do.


Maybe your thesis that these lists are BS is a bit strong especially given that they are recommendations.



RogerWaters said:


> Thus, I don't think this proves TC lists are completely pointless. I have discovered much great music using them. However,


You have discovered much great music using them so perhaps you could characterize them as quite useful rather than pointless. Again, the lists are recommendations. For you and many others, they were definitely beneficial and produced the desired effect.



RogerWaters said:


> I think what this shows is that the rankings are completely erroneous. If people don't take them seriously in the first place, then fine, you've nothing to see here. However, if you do, then arguably you shouldn't.


Some people have pointed out that members voting in your polls could be largely different than the members who participated in the projects you listed. That alone could produce the results you mention. Others have said that pointing out issues with a pair of works out of 100 or so hardly demonstrates a completely erroneous result.

More importantly, the completely different methodologies can easily create the apparent contradiction in rankings. There are several ways that work A could be preferred to work B in head to head polling but that, using the alternative methodology, work B would be ranked above work A. The lists to which you refer are not stating that the participating members prefer one work to another but rather that the cumulative ranked recommendations of the participating members rank one work above the other. Those are two different conclusions.


----------



## Nereffid (Feb 6, 2013)

RogerWaters said:


> I think what this shows is that the rankings are completely erroneous. If people don't take them seriously in the first place, then fine, you've nothing to see here. However, if you do, then arguably you shouldn't.


Pretty much the only people who take the rankings _seriously_ are those who complain about them.


----------



## RogerWaters (Feb 13, 2017)

Nereffid said:


> Pretty much the only people who take the rankings _seriously_ are those who complain about them.


I like this very much


----------



## RogerWaters (Feb 13, 2017)

mmsbls said:


> Maybe your thesis that these lists are BS is a bit strong especially given that they are recommendations..


I admit I was being intentionally polemic. However, I think the rankings are mostly erroneous as rankings, especially when you get down to, as MacLoad says, sub ten rankings.



mmsbls said:


> More importantly, the completely different methodologies can easily create the apparent contradiction in rankings. There are several ways that work A could be preferred to work B in head to head polling but that, using the alternative methodology, work B would be ranked above work A. The lists to which you refer are not stating that the participating members prefer one work to another but rather that the cumulative ranked recommendations of the participating members rank one work above the other. Those are two different conclusions.


What is the significance of the difference?


----------



## Guest (Jul 22, 2020)

RogerWaters said:


> I admit I was being intentionally polemic. However, I think the rankings are mostly erroneous as rankings, especially when you get down to, as MacLoad says, sub ten rankings.


How can these rankings be 'erroneous'?


----------



## Enthusiast (Mar 5, 2016)

I do wonder why you expected the lists to be reliable indications of universal preferences. Surely it was clear that they could not be?


----------



## BlackAdderLXX (Apr 18, 2020)

I'm really sorry to hear about this. Over the past few months I've been getting the lists tattooed on my chest (backward so I can read it in the bathroom mirror) and now, just as I've had my last session at the tattoo shop I find that these lists aren't definitive?!? This is very disappointing. I guess I'll see if I can get the tattoo covered up with a picture of Furtwangler. He's always at the top of the list.


----------



## BlackAdderLXX (Apr 18, 2020)

Enthusiast said:


> I do wonder why you expected the lists to be reliable indications of universal preferences. Surely it was clear that they could not be?


I always looked at them as if to say, "Some people like this work. Maybe I might like it." The numbers are irrelevant to me. And I know it's all fake news because Beethoven's 5th should be at the top and it isn't.


----------



## hammeredklavier (Feb 18, 2018)

BlackAdderLXX said:


> I guess I'll see if I can get the tattoo covered up with a picture of Furtwangler.


Yes, with a big swastika in the background


----------



## JAS (Mar 6, 2013)

Have 60 Minutes and the New York Times been notified? (This scandal would make for a nice break from the political and COVID news)


----------



## hammeredklavier (Feb 18, 2018)

UniversalTuringMachine said:


> The title of the post is sensational and insulting to people who contributed to the TC lists


Would this have been less offensive?:
"TC rankings exposed as B"


----------



## consuono (Mar 27, 2020)

I look at these polls etc as conversation starters, not attempts to be stone-cold accurate. It's no big deal.


----------



## Strange Magic (Sep 14, 2015)

Final nails in the coffin of the "objective" excellence/greatness/superiority of this or that work of art. Voting and polls and opinions are at the heart of esthetics, truth be told.


----------



## science (Oct 14, 2010)

Roger, my dude, you just need to vote in our threads.


----------



## philoctetes (Jun 15, 2017)

I just took another poll and got different results too... 

I don't play games or look at the results but having a recent interest in British composers I threw my vote for Bax into the ongoing poll on another thread... which only included 5 choices for the entire history of British music... what good is such a poll when it excludes Walton, Delius, Tallis, Byrd, Ferneyhough, etc... 

it's like going to the voting booth and finding that one of the top candidates has been left off the ballot... mmm, now just where might that happen...


----------



## Ethereality (Apr 6, 2019)

Another friendly reminder that Talk Classical has a big poll *here* that will keep collecting results throughout the years! List your top works of all time and they will show up in the official list. You can submit for instance just 10, 20, or even 100. You can erase or change your list any time. We're getting more and more submissions.


----------



## Bulldog (Nov 21, 2013)

hammeredklavier said:


> Would this have been less offensive?:
> "TC rankings exposed as B"


I don't appreciate my avatar being associated with your postings - cut it out.


----------



## senza sordino (Oct 20, 2013)

As the facilitator of one of those TC rankings (Piano Trios in 2016/2017), I assure you I did everything I could to legitimize this ranking. I solicited the help of a qualified statistician, I bought some time at the local university supercomputer for a run time of about nine days before I had my final results, and then had the results certified by a prestigious accounting firm. 

I'm kidding of course, and I don't mean any real offense, but this is my initial reaction. 

If we were to run the same TC ranking of piano trios with the same people, we wouldn't get the same results. There were 125 piano trios in the final results. And there were many more honorable mentions. At best, I say you could probably divide the 125 into quintiles. If we were to run the TC ranking again, with the same people, the top 25 might be very similar but a different order, the next 25 might be similar but in a different order, 51 through 75 the same again etc. 

The wisdom of the crowd can guess the weight of a cow fairly accurately by averaging all their guesses, but that is because the cow has a measurable weight, it's not an opinion. The wisdom of the crowd cannot rank piano trios because there is no true rank, it is an opinion.


----------



## juliante (Jun 7, 2013)

KenOC said:


> What these lists prove is that too many TC denizens are driveling cretinous idiots with the musical taste of diseased marmots. Comparing Beethoven's and Sibelius's 4th Symphonies is a case in point: It will be immediately obvious to any normally-intelligent and mentally-fit person which is superior. But the brain-dead inbred imbeciles in this forum, many with family trees that look like telephone poles, will get it wrong every time.
> 
> So yes, we have some problems with lists (and with polls for that matter). But nothing is to be done, so it's probably best just to smile and act as if all the others are fully as brilliant as are we ourselves.


I say Ken! Don't think I've seen you so exercised.


----------



## Enthusiast (Mar 5, 2016)

philoctetes said:


> I just took another poll and got different results too...
> 
> I don't play games or look at the results but having a recent interest in British composers I threw my vote for Bax into the ongoing poll on another thread... which only included 5 choices for the entire history of British music... what good is such a poll when it excludes Walton, Delius, Tallis, Byrd, Ferneyhough, etc...
> 
> it's like going to the voting booth and finding that one of the top candidates has been left off the ballot... mmm, now just where might that happen...


But the results so far do seem to show that more than 5 would not have changed the result. Yes, lots of us know, like and value the composers you name but it seems like but it seems we would not have voted any of them over the names that were listed. The quick poll was looking for our various single favourites among English composers and was therefore more about us as a forum than about the composers. Your top candidate was not there but if he had been I don't think he would have beaten the listed ones in the popularity among members stakes. If the poll had been looking for merit I do think that a good few more earlier composers and contemporary composers would have been needed.


----------



## larold (Jul 20, 2017)

Polls are popularity contests. What they "expose" are the collected preferences of the mass that participates. 

I've seen more than one American poll saying the favorite piece of music is Gershwin's Rhapsody in Blue. 

The Gramophone reader poll of best recordings of the 20th century listed Elgar's Cello Concerto No. 2 behind Solti's Ring.

Time and exposure also influences outcomes. Until this century I never saw a poll where Mahler fared well. Now it happens all the time.


----------

