RegisterDonateLogin

The perfect complement to Wookiee Cookiees.

Welcome Guest Active Topics | Members

The real reason the Squad Ratings on this Site are Useless Options
billiv15
Posted: Thursday, June 3, 2010 10:27:15 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 4/4/2008
Posts: 1,441
Actually Echo, you proved my point with another example. What I started with was just one example of a squad that was being rated high without solid reasons (actually two separate versions of the same squad to be precise). You gave another example of what makes the ratings useless.

And to deal with that, Shinja made a "non-dci squad" rating option. So to me, people who are still rating squads like the best "Revan I've ever seen" squad with a 10 for DCI play are doing so without merit (obviously they may not know there is another option as well - but I don't consider ignorance a positive excuse generally - even if it is a valid explanation).

Further, I think some of you are misreading my posts to say that someone has to play a squad many many times. What I said was, I won't give above an 8 for a squad that hasn't been played. There are plenty of ways to "prove it" as it were. But be serious for a moment guys. The ratings on here have never, ever reflected what is actually winning the big tournaments before those events. The tournament squads that do make it high on the list, tend to do so afterwards - so clearly the community already values "proving it" as I have stated.

Unless you are willing to argue that an 8/10 isn't a strong rating, in which case I would ask how you decide what to rate high. I would also suggest, you are likely a person who doesn't help the rating system, as odds are you are rating 50% of the squads you see with an 8+ which actually does nothing positive at all for anyone. You might as well rate everything with a 3.

I consider relative rankings, which is what a scale system is, to function like a bell curve. Most squads should get between 4-6. That's how I rate. A good squad gets a 4-6. A squad that is really good gets a 7-8, but that has to be about 10% or so. Only the final 1-3% should get 9-10 ratings - and the only way to show a squad deserves that kind of consideration is to in some way show it.

Big events are of course the easiest way to do so, but let's not pretend its the only way to make me look like I'm being biased. Vassal games work, local tournaments work, heck, post some blasted play reports with your friends on what you beat, how it went and so on. No one has an excuse not to show their work. In my opinion, and I think it's pretty valid, not even doing the basic amount of work shows you aren't really interested in making a truly top squad - as that's what ratings should be about. After all, I can tell you what about 90% of the top squads are at any given moment, but that's only because I play enough to have that knowledge. It isn't a biased towards top level players, it's simply a matter of work. Making a squad of random characters is about 30% of making a meta winning squad. If you want me to be fair, I will start using that in my ratings. A squad that doesn't show play testing or any results, will now max out at a 3 for me, as you've at best done 30% of the work required to truly have made a top squad. Now I'm not really going to do that, but I think you can see the point.
saber1
Posted: Friday, June 4, 2010 1:10:54 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 2/14/2009
Posts: 1,450
Location: At the controls
FlyingArrow wrote:
The ratings are a popularity contest. The site should have a separate place where tournament-winning squads can be highlighted so there a clear view of what the top squads are. They're mentioned in the play reports, but there's no central place to list them. Instead of a top-5, I'd be more interested in seeing...

All of the regional squad winners and runners-up.
Top 4 squads from every GenCon.


This sounds like a good idea.

@billiv15: You seem to be focused entirely on ratings as they pertain to competitiveness. There are many times ratings are based on creativity and "fun factor" rather than the squad's ability to dominate all others. Is one method necessarily better/worse than the others? As with anything on the internet, take squad ratings with a very large grain of salt.
billiv15
Posted: Friday, June 4, 2010 2:03:16 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 4/4/2008
Posts: 1,441
saber1 wrote:
@billiv15: You seem to be focused entirely on ratings as they pertain to competitiveness. There are many times ratings are based on creativity and "fun factor" rather than the squad's ability to dominate all others. Is one method necessarily better/worse than the others? As with anything on the internet, take squad ratings with a very large grain of salt.


It should be obvious that I take them with a grain of salt. The problem is, that the vast majority of the community does not. People consistently look at ratings and expect them to mean something, whether they actually do or not. I don't think anyone who has followed this board, read comments from people who use squads from here, and so on and would have any other understanding of the issue.

And as for whether or not one is better than the other, well absolutely true, one is and I will tell you why.

There is absolutely no point in having a "rating" system if all it means is whatever I want it to be. In fact, it's an absolute detriment to the community if that's what it is (I will argue that it's not that at all however already, and I'm trying to improve it at what it is.) What's the point of doing a rating, if what you are rating on is "fun". "Fun" is entirely arbitrary, with 0 consistency, and completely dependent on the rater. As a completely different person, I do not care what you think is fun, and I certainly have no interest in what you rate something on a scale rating system if that's how you rate. What's the point of it? Creativity is the same thing. One person's creativity is another person's totally boring. Who cares. In fact, you should represent your appreciation for fun and creativity in the comments section, not as a rating.

To add to this, people who rate this way are actually a problem to the system. Because they counteract the ratings that the majority are making. I understand there will still be difference of opinion in terms of what is really competitive or not, and that's fine. But people rating based on "creativity and fun" just throw the entire system out of whack and in fact, do harm to it. Hence my original title of why the squad ratings are "useless" applies. With these things going on, the ratings have 0 chance of meaning anything at all ever.

Is that what you all really want? I will make a poll to find out. I think if the majority of people are honest about it, they want the ratings to at least matter to a degree, and probably already use them in some way as a guide. But let's see what happens.
RannKonnar
Posted: Friday, June 4, 2010 2:12:41 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 12/17/2008
Posts: 761
Location: Indianapolis, IN
Great.
More arguing about stupid ratings.
Glare
Why cant people just shutup about stupid ratings, and get along on this SWM we are on.
Falconjo
Posted: Friday, June 4, 2010 2:54:41 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 9/9/2009
Posts: 106
hehe now im just going to rate everything a one because i hate fun! Flapper
Sithborg
Posted: Friday, June 4, 2010 3:56:16 AM
Rank: Moderator
Groups: Member , Moderator, Rules Guy

Joined: 8/24/2008
Posts: 5,201
There, the thread is cleaned up, happy? Keep it on the subject, and not the posters please.
DaddyCool
Posted: Friday, June 4, 2010 3:56:17 AM
Rank: Wookiee Elite Warrior
Groups: Member

Joined: 12/5/2008
Posts: 19
I think it's impossible to fix this system unless you provide a generic definition for the numbers, and even if you'd do, there will be disagreements.

I think the best idea would be a separate subforum or sthg where people can upload tournaments results and such with the squads.

I, for one, am really interested in tourney results, and find a bit disappointing that outside the top8, you rarely get any squad infos even from Gencon, which is supposed to be the biggest event in SWM.

Just my 2 cents.
Falconjo
Posted: Friday, June 4, 2010 4:23:38 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 9/9/2009
Posts: 106
DaddyCool wrote:
I think it's impossible to fix this system unless you provide a generic definition for the numbers, and even if you'd do, there will be disagreements.

I think the best idea would be a separate subforum or sthg where people can upload tournaments results and such with the squads.

I, for one, am really interested in tourney results, and find a bit disappointing that outside the top8, you rarely get any squad infos even from Gencon, which is supposed to be the biggest event in SWM.

Just my 2 cents.


i agree, i love local tournaments where you see all sorts of squads win, at my LGS we have some crazy squads show up and some of the most creative win..

ex. we had a gungan bounty hunter squad win, a force immune missiles squad win, even a sith sorcery squad took second one week.

I love diversity and creativeness in squads, dont get me wrong i respect the top gencon squads because they win, and who doesnt like to win? But I love to see the variety that this game gives us.
And I believe that a squad shouldnt suffer in the "ratings" because it wouldnt be able to compete at Gencon.

NO squad is perfect, there is not a squad out there that will win everytime against every opponent. So no squad would deserve the perfect rating of 10, because it wouldnt win every game.

The last thing I want to say is if people want a "competitive" section to ratings, then lets meet in the middle. Lets have a competitive scale from 1-5 and a creative scale of 1-5 If a squad has proven to win and is proven to be quite creative it would have that "10" that people seek.
Echo24
Posted: Friday, June 4, 2010 5:06:16 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 9/30/2008
Posts: 1,288
billiv15 wrote:
Actually Echo, you proved my point with another example. What I started with was just one example of a squad that was being rated high without solid reasons (actually two separate versions of the same squad to be precise). You gave another example of what makes the ratings useless.


Oh, I know. That's because I don't disagree with you. I too believe that the ratings are useless in the sense that a squad being rated high or low doesn't really mean anything. I was just pointing out why I believe they are useless; that is, since there is no rubric for rating squads, people will just rate squads all willy-nilly and subjectively. And under the current system, I don't think there is a problem with that. It's just a fun aspect of the site and isn't meant to be taken seriously.

Now, if there was a specific "competitive" rating to go along with the "general" rating, I would join you in the fight to make the competitive rating meaningful in some way, and I think this is a good idea. But the "competitive" rating shouldn't be the only rating; there isn't anything wrong with rating things high just because they're neat. I know that you don't rate squads that way, but in the current system, the way you rate squads isn't necessarily better than the way anyone else rates squads. Which, yes, makes the rating system pretty useless as a true measure of "good" or "bad" squads. It's just a fun little tool that should be taken with a grain of salt at best.
billiv15
Posted: Friday, June 4, 2010 5:42:51 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 4/4/2008
Posts: 1,441
Echo24 wrote:
It's just a fun little tool that should be taken with a grain of salt at best.


I don't totally disagree with you. But I just want to point out that this statement has been made at least 1000 times on this board, and still, people care more than a grain of salt. I've personally said that multiple times to people, yet it keeps coming up (recognize I brought it up this time obviously). But the point remains, it's quite clear that this is not really true, no matter how much we say it. People still associate rating with some kind of meaning.

And that to me, ends up being misleading and a disservice to the site. I believe the ratings should be removed completely, or reorganized. Doesn't matter to me what they represent in the end, just as long as it's something. And I honestly don't care that much, just wanted to discuss it with people and see what we all thought about it. A bit bored at work the other day :)

Two ratings might be a better way to go, but I'm really not sure why we need the second.

Can someone in the "rate high for fun" department explain to me why they want a squad rated high? What does a rating mean when all it represents is someone liked it.

Actually, the correct solution I think would be to do the following.

Adopt a set of guidelines to be stickied on the forum in how to rate, and how to scale the numbers that the community accepts. People will have differences in opinion, and that's great, because they will all be using a true rating methodology that leads to a true bell curve rather than the willy nilly garbage we have now.

If you can get even 50% of the squad raters making an effort, the ratings will improve dramatically.

Second, you make a "like it" and "love it" button (or whatever you call it). "Like it" scores one point, "Love it" scores 2 and the squads with the most points make the "Creative Fun Super Top 10". Then you can ignore squad size, legality, etc.

Giving a 9 to a squad I just like makes little sense to me. What is that based on? Why would I do it? It's actually completely illogical if you think about it. I mean they don't just give out pies at the bakery down the street because the waitress thought I was cute walking by. Seems kind of the same thing to me :)
shinja
Posted: Friday, June 4, 2010 8:55:11 AM
Rank: Administration
Groups: Administration

Joined: 10/2/2008
Posts: 351
Location: Kent, WA
Right now ratings are purely a popularity thing. By giving a squad a high rating, you are saying it deserves to be noticed. By giving it a low rating, you are saying it does not deserve to be noticed.

Squads with a lot of ratings (high or low or both), are squads that a lot of people have looked at, and that a lot of people had an opinion about.

If a squad gets a lot of ratings, and manages to keep a pretty high average, then it generally is a squad that deserved to be noticed (for whatever reason). I say generally, because when people artificially inflate the ratings for a squad, they mess up this system.

I'm not saying that this is how the system SHOULD work, I'm saying it's how it DOES work currently.

I'm hesitant to clear the ratings and/or comments when someone updates their squad, because it gives the squad creator the ability to remove unfavorable ratings or comments by taking out an Ugnaught and re-adding it or something equally meaningless.

I like the idea of having a section that features tournament-winning squads. Unfortunately, at this point, it would be difficult to validate local tournament winners. But for now, if someone wants to compile a list of winners or high-placing squads from the major tournaments (dating back however far you want to go), I'll add a section for them.

Keep posting ideas for how to improve the system. Once I get the new Squad Builder in place (hopefully in the next update), I'll see what I can do about the ratings.
RannKonnar
Posted: Friday, June 4, 2010 9:30:31 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 12/17/2008
Posts: 761
Location: Indianapolis, IN
I agree.
I dont mind ratings.
greentime
Posted: Friday, June 4, 2010 9:50:48 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 5/23/2009
Posts: 177
billiv15 wrote:
What we really need is for the community to take ownership of the ratings and rate with at least some consistency. Not everything can be fixed, but the reason I posted this example was because it shows quite blatantly that people are not even doing the bare minimum of thought and reading before rating these two squads.

The problem is, that the vast majority of the community does not. People consistently look at ratings and expect them to mean something, whether they actually do or not. I don't think anyone who has followed this board, read comments from people who use squads from here, and so on and would have any other understanding of the issue.


So "the community" is a bunch of irresponsible people who don't think through their ratings and just throw some numbers around. The same group of people, the ones who made all the bad ratings, also believe that the ratings are the best indication of how good a squad is. Quite frankly, it seems very unlikely that both of these things can possibly be true.

But hey, thanks for insulting us. That really helps move things forward. What is the point of this post? You, Bill, do not like the way the rating system has been used. Ergo, the rating system is broken, and "the community" needs to come together and bring it up to your high standards. Do you even realize how high-handed and condescending your posts in this thread have been?

I agree that the rating system is not as useful as it could be. But the best way to make it better is not to show up one day and tell everyone that they suck and need to get with the program and do it your way, the one and only correct way. A section exclusively for squads that have placed high in recognized tournaments is a very good idea. More ideas like that and less castigation, please.

billiv15
Posted: Friday, June 4, 2010 11:35:59 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 4/4/2008
Posts: 1,441
I really don't know why it's so hard for some of you to believe that "I don't care about squad ratings personally". So to say this is all about me, or any other nonesense is simply that. I'm sorry for bringing up such a delicate subject (the fact that my posts which have 0 emotion in them) are being read as being my personal opinions and desires is quite hilarious actually.

I simply thought since the trolls seemed to be gone to a degree, it was something we as a community could work on improving. I don't care if you use my solutions or not, all I really wanted to provide was a good discussion and my personal analysis of the issues. Just because you don't like my analysis or whatever, doesn't mean I'm being "insulting" and "condescending".

If some of you will get past the "Bill must be insulting me" comments and read what's going on in the thread, there have been several good ideas and it looks like Shinja is on board with what seems to be the most useful and desirable - making a tournament winner type subsection of squads. I think that's a great solution to a problem that has been brought up at least 15 times in the last couple of years.

What you quoted was simply a matter of reality. I did not say the same people were in both categories. But I'm sorry, I've seen way too many posts of people like you saying "ratings don't matter" - I've posted it too many times to count to help people as well, and I've seen way too many posts (by other posters) who have referenced bloomilk ratings in a discussion. One shows that some of us think they are crap, and others think they mean something. Under the current system, it's not possible that they mean anything other than popularity. But the reality is that many posters will be, have been, and still are mislead by it being a ranking.

If you can't see the problem in that, then I don't know what else to say and will bow out. It's a kind of false advertising really. You advertise a ranking system, but the system in place doesn't give the expected results in a way that many posters have shown us with their posts read the answers. Rankings are supposed to tell you something, and most people don't interpret that on first look to mean, "personal favorites".
njarnagin
Posted: Saturday, June 5, 2010 2:05:07 PM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 12/26/2008
Posts: 168
shinja wrote:
Right now ratings are purely a popularity thing. By giving a squad a high rating, you are saying it deserves to be noticed. By giving it a low rating, you are saying it does not deserve to be noticed.


I think that's a good summary of what ratings mean to me. How well is the rating system functioning according the "popularity contest" definition? That's hard for me to say.
mercenary_moose
Posted: Saturday, June 5, 2010 2:29:27 PM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 12/11/2008
Posts: 1,122
greentime wrote:
billiv15 wrote:
What we really need is for the community to take ownership of the ratings and rate with at least some consistency. Not everything can be fixed, but the reason I posted this example was because it shows quite blatantly that people are not even doing the bare minimum of thought and reading before rating these two squads.

The problem is, that the vast majority of the community does not. People consistently look at ratings and expect them to mean something, whether they actually do or not. I don't think anyone who has followed this board, read comments from people who use squads from here, and so on and would have any other understanding of the issue.


So "the community" is a bunch of irresponsible people who don't think through their ratings and just throw some numbers around. The same group of people, the ones who made all the bad ratings, also believe that the ratings are the best indication of how good a squad is. Quite frankly, it seems very unlikely that both of these things can possibly be true.

But hey, thanks for insulting us. That really helps move things forward. What is the point of this post? You, Bill, do not like the way the rating system has been used. Ergo, the rating system is broken, and "the community" needs to come together and bring it up to your high standards. Do you even realize how high-handed and condescending your posts in this thread have been?

I agree that the rating system is not as useful as it could be. But the best way to make it better is not to show up one day and tell everyone that they suck and need to get with the program and do it your way, the one and only correct way. A section exclusively for squads that have placed high in recognized tournaments is a very good idea. More ideas like that and less castigation, please.



(fist-bumps greentime)

billiv15 wrote:
I really don't know why it's so hard for some of you to believe that "I don't care about squad ratings personally".


I dunno... maybe it's because you're the only one here wasting your time typing up huge eight-paragraph-long posts that rail against a strawman for everything you perceive to be wrong with a rating system considered by all to be completely irrelevant?
thodafett
Posted: Sunday, June 6, 2010 2:04:54 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 8/5/2008
Posts: 173
Location: Nampa, ID
I think it would be to the advantage of the tournament players to NOT have a system that tells the whole world what the best squads are. I know skill has the largest say in determining a winner, but being a dice game, there is an element of luck involved, and well I would think tournamnet players like to have every advantage possible going into a tournament. Obviously any net-decker can come on here look at the 'top" squads of the day and go on their way, go to the tournament and have their squad smacked around and go home empty handed. I have never seen in the years I have followed this game, the Gencon Champ's squad as being listed as the top squad before the Gencon tournament, I could be wrong about that. There are roughly a 100 Gencon Championship players, and perhaps another 100-200 highly competitive players that post here, Gamers, or back on Wizards. How many thousands are registered on each of these sites that obviously aren't competitive players or don't only care about the current meta? I would suggest coming up with a squad rating system that would be to the satisfaction of the top tournamnet players. I hate to say it, but I don't think Bloomilk is the site for that to happen, judging on the responses I have read in the last few threads on this topic it would seem many of the posters are just as interested in themed squads, fun squads, and competitive squads. It is hard to separate them, when there isn't an option to do so. Unless there are huge changes, which may or may not happen, I don't see how we can please everyone.
TreebeardTheEnt
Posted: Sunday, June 6, 2010 7:50:23 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 6/21/2008
Posts: 193
Location: Dallas / Fort Worth
Bill, I just noticed that most of your squads you give your self a 10. If rating doesn't matter (and I believe you when you say it does not) why are you rating your squads so high? They are not all top level squads that would win regional or Gencon...

I'm not trying to start anything, I was trying to improve my squad building skills and thought that your rating of your squads would help with identifying combinations and concepts and build from them.
billiv15Booster
Posted: Sunday, June 6, 2010 9:32:28 AM
Rank: Caamasi Noble
Groups: Member

Joined: 2/6/2010
Posts: 4
Can't log into my normal account at the moment, so I will respond using my booster league account.

The answer is simple. I started the practice of giving every squad I make a 10 to counteract at least one of the 1-bombs I would normally get. For about a year on here, every squad I made would get 2-10 1s within a matter of days. I figured no reason to rate my squad rationally because a 1 + 10 = 2 votes of 5.

I've kept doing it because I still get 1-bombed (again, please remember I don't care) and maybe more importantly I find it really hard to rate your own squad. Personally, I don't even know how to rate my own squads. I think they are all 10s and have no idea how someone could possible be even the least bit partial on their own creations. With that said, I'm did go through my squads a while ago and rated nearly all of them as accurately as possible. I just don't bother with the new ones and simply click 10, figuring others can judge better than I.

You guys are free to rate them as you please, I don't care. I've already mentioned how I rate squads in general. For a brief recap to a long post I made a few months ago:

1-3 = squad that needs serious work for competitive play (and by that I mean competing at a local store). Usually these refer to squads that are lacking something really significant, like door control, etc.

4-6 - Good squads that are generally well built. They might have a small flaw here or there, or are less than optimized, etc. But generally in the hands of a good player, these will do well locally.

7-8 - Top level squads that should compete at the highest levels in some way or another. A 7-8 might have a very minor flaw, or something I would shift here or there, but generally are already top tier.

9-10 - Reserved for tournament winning squads, Gencon champs, well built regional or big tournament winners, or in some cases squads capable of winning such events even if the creator isn't able to play in such events. I often change a rating from 7-8 to 9-10 if a squad I previously rated wins a big event in the hands of it's creator, or someone who references said person in their tourney report.
trappedslider
Posted: Sunday, June 6, 2010 11:07:50 AM
Rank: Advanced Bloo Milk Member
Groups: Member

Joined: 12/4/2008
Posts: 371
Location: Roswell,new mexico
billiv15 wrote:
[Who cares. In fact, you should represent your appreciation for fun and creativity in the comments section, not as a rating.

To add to this, people who rate this way are actually a problem to the system. Because they counteract the ratings that the majority are making. I understand there will still be difference of opinion in terms of what is really competitive or not, and that's fine. But people rating based on "creativity and fun" just throw the entire system out of whack and in fact, do harm to it.


Thank you for telling me to Stop having fun.

I don't take Minis as serious business but i did post my two squads one was for semi-compertativeness and the other a squad of Stormtroopers and haloveers was pure fun and creative and so I do two things when rating i frist look at how creative the squad is and then i look at how "competaive" I think it is.
Users browsing this topic
Guest


Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Main Forum RSS : RSS

Bloo Milk Theme Created by shinja
Powered by Yet Another Forum.net.
Copyright © 2003-2006 Yet Another Forum.net. All rights reserved.