Are star ratings in reviews useful?

[atani]

If star ratings are meant to give feedback on how good something is, then the Hub results shows that things are very good or very bad, with little in between.
Of over 850 resources pages:

• not even 4 pages for all 1 to 3 stars ratings
• around 20 pages for 4 star ratings
• around 310 pages for 5 star ratings
• around 500 pages have no rating and review

If reviewed items are overwhelmingly 5 stars content, then almost everything is excellent?
The star ratings end up not given as a show of how good the content is, they're given for all kinds of reasons, whatever they may be, but all those different reasons have to be translated to a single number.

You can only rate if you write a review, and you can only write a review if you rate it

The review textis really special for the creator, it shows that the reviewer made the effort to provide feedback, this is very motivational. Even if it's a review showing it's bad for some reason, it's still feedback, and that is really hard to receive.
For users the review shows if something is good enough. The stars are meaningless because it's probably going to be a 5 or 1. The text will be much more meaningful on why and how the person reviewed it. Of course, not having a review doesn't mean it's not good, just that getting feedback is difficult.

My proposal is to stop using star ratings. They give unrealistic impressions of how good something is because a decent enough and free asset with less than a 5 star rating is seen as crap in a world of 5 stars, so the automatic 5 because it's free and not bad. Or at least not be mandatory to pick a star rating when writing a review.


Would be great to hear more thoughts on this. If you have UX/UI experience or data analysis related to the topic that would be amazing, because this is just my biased viewpoint.

Other related topics:

What about the positive stuff?

Up to now we've all poked at "issues", but I wanted to take a minute to say this hub site is fabulous. So while we all have (and will have) ideas and gripes, overall this is amazing. I love: - Free v.s. Paid categories - Ability to easily find resources by type and authors - Ability to easily...

hub.virtamate.com

Stars rounding

It's been a while and I really didn't want to make a fuss because, you know, it's not that important... but I think rounding may be better than flooring here :) I knoooow I'm being self centered and all, but I felt that I may not be the only one who got a 4 stars review and went "nooooooooo!"

hub.virtamate.com

 

[Acid Bubbles]

It's interesting that you have grabbed that data. I know personally, I tend to either rate 5 stars for stuff worth other people's time (and the creator's attention) but I don't like using stars to provide less-than-perfect feedback, since it's permanent. I'm sure that having the name attached to a bad review also makes people with any visibility think twice about doing that. I know negative feedback can be very damaging, and platforms like StackOverflow will actually make you "pay" some of your reputation to vote something down to ensure negative votes are actually a serious thing. I'm not suggesting that, but I also think that weird and taste-specific content will be hurt by "easy downvotes". So.... very hard to say what's the right way to go. Steam also "forces" you to write some text alongside your review along with your name, that's another way to try and improve the quality of reviews, even if it's just a few words.

Right now there's star ratings (which I do think provide the most value right now), reactions (which indicates how "attractive" the content is from the thumbnail to people), and downloads (which shows how popular something is but not if it was worth the download). I kind of feel like there's no "clearly superior" alternatives, since having just a "good / bad" rating wouldn't make a very big change to the overall reviews landscape IMHO.

Maybe one thing that _could_ be done is not to show ratings for something with less than 3 reviews (just a guess, would have to check the data), but since the community is small, a lot of resources will actually only get very few ratings, it'd be a shame to ignore them.

Long story short, I have no idea. Also, it's funny that you actually referenced two threads I started, I guess I care too much

 

[atani]

Also, it's funny that you actually referenced two threads I started, I guess I care too much

We should all care, even if just a little

I would like to mention that the review is made of 2 feedback channels, the star rating and the review text. Currently to make a review you need to rate and write something, which in my view this coupled nature influences not only the rating disparity but also in review reluctance. It's just a opinion, I wish I had expertise in research of this topic, but I don't.

I tend to either rate 5 stars for stuff worth other people's time (and the creator's attention) but I don't like using stars to provide less-than-perfect feedback

From the number of pages with 5 star ratings there's pressure to not give less than 5 stars or not make a review at all. There is plenty of assets that may have some issues to fix, where a 4 star would fit well until those were resolved, but it will likely not get reviews because giving 4 stars kind of feels like it's 1 star. Unfortunately those are the ones, where 3 and 4 stars would be suitable, that need the reviews the most to incentive the creator to improve or correct existing faults.

Right now there's star ratings (which I do think provide the most value right now), reactions (which indicates how "attractive" the content is from the thumbnail to people), and downloads (which shows how popular something is but not if it was worth the download). I kind of feel like there's no "clearly superior" alternatives, since having just a "good / bad" rating wouldn't make a very big change to the overall reviews landscape IMHO.

Absolutely, there's no superior system or will ever be one. I like a combination of systems, as you said they give different information which then allows a person to potentially have a better idea if the asset is whorthwhile for them. However, more is not always better, because with each system you get their pros and cons, and if the cons are too strong for the environment it's used it might be time to reconsider having it as currently is implemented.

In the way it's implemented and the environment it exists in, the star rating is currently showing that something reviewed with 5 stars can be: mehh but it's free, it's ok, good, excellent, a must have; and they're all the same rating. When the rating is the same for all these, then the rating system pros are unlikely to justify the cons.

While I kind of distrust the star rating usefulness, one thing that I think would help would be to not have the rating as mandatory. If you want to make a review but do not consider that the material is worthy of 5 stars, not having to rate it would still allow you to write the review in a sincere way without "hurting" the creator with less than 5 stars. This might lead to a increase in reviews and more honest reviews too.

Side note:
Just today I saw someone giving a 1 star to someone's asset because there was a mistake in the packaging. As I see it, a corteous approach would be to mention that in the discussion section instead of the review area, but I don't know the reviewer even knows there's a discussion section, could be why the review like this.
Relating the topic question, I wonder if the same review was made but there were no star ratings, would it have such a strong impression like it has with the 1 star given? Even just not having them as mandatory, I wonder if the reviewer would skip the rating and would prefer instead just to leave that note.

 

[Juno]

Very hard metric to deal with considering 98% of the userbase doesn't rate or review anything as there's literally nothing making that happen. Perhaps if there were a requirement to rate and review a % of the resources you download or be prevented from downloading anything we might see some better participation. I have long thought you should not be able to download anything from the hub without a logged in account.

 

[atani]

One reason I ask that question is because I suspect there's some potential reviews that are not made because star ratings are mandatory. Reducing that barrier would possibly increase the reviews, especially for ok material due to the perceived "less than 5 stars is shitty" impression.

 

[Vam_SlesaR]

You know, I think the stars don't reflect the quality of the content. It would be great if, when uploading content through the internal VaM interface, it was suggested to anonymously evaluate the quality according to several criteria. For example, the evaluation window at the next VAM startup. And to avoid cheating, you can get hardware ID for uniqueness

 

[HolySchmidt]

For me it's like this: there's so much good stuff for free. If I really, really like the content, I rate it and give it 5 stars. Cause it's free and good.
If I don't like it, I don't rate it.

So with this system, the more ratings content gets, the better it is. Plus: the creator gets a nice response and is happy.
I think if more people would be really honest with every rating, we would have less content cause a lot of people would be pissed, and I could totally understand it.
Like "I spent hours to give you some stuff for free and you give me only 3 stars?".

I had this kind of response one time: the textures on a clothing looked like a 256-colour texture, not that good. I gave my honest response (3 stars, bad textures) and the creator was really pissed. What to do?
Good free content: 5 stars. Not that good content: no rating.

And for all these non rated content: it's just too much. You can't rate all the good stuff and a lot of it is not THAT good. Additionally, like already been said, the "community" is pretty ungrateful. A lot of people just download and enjoy. No "thanks", no nothing. "Internet-free-stuff-phenomenon".

 

[Lytanshade]

The only useful stats would be something like VAM polling which plugins people are using and reporting that back to the hub. But that's probably a bit too invasive. The internet is synonymous with the saying "one bad apple can spoil the barrel", there'll always be the miserable or trollish personalities that can't help themselves.

I'm probably typical in that I'm actively working on a few scenes/projects but I check the Hub daily and download anything that looks even slightly promising for future works. And, unless it's from a creator that I know produces excellent work, I don't comment or upvote unless it's something I start using and think 'boy, this is good!'. And that may be months after downloading it. And any emotional connection between the item/creator on the Hub and in game is lost. Although, if it's something I'm using and the creator uploads something else I'm much more likely to upvote it sight-unseen.

Really though, the only useful info is a 'trusted' user leaving a comment like, "absolutely must-have addon". I mean if Acid bubbles comments that it's an awesome plugin then that's good enough for me.

 

[Lytanshade]

One useful statistic that might be trackable is: Number of times included as dependency on Hub.

Sorting by that would likely raise the cream to the top of the results.


The Hub tracking and sending out a list of everything you've downloaded each month with a reminder (and quick links) to review/rate would make a big difference as well I guess.

 

[everlaster]

or this shit, look at this douchebag named @D91d
View attachment 136095

This dude shared a wonderful resource, made 17 updates to it (!), shared it always for free, and has to deal with morons like @D91d because of having the impertinence of making you do an extra click and consider supporting him make more stuff in case you like his assets. What's he supposed to do? Delete patreon and just slave away assets for vam hub stars?

Let us start banning people like that. If they don't like patreon links, they can skip the items that don't have the green Hub-Hosted. Or maybe make the label bigger for @D91d to see it more clearly before he wastes his click. Imo stuff like this shouldn't be tolerated

There's nothing particularly wrong about D91d's review text itself - if that message had been posted in the resource's discussion rather than as a review, it'd have been fine. But because it's a review, you have to specify the number of stars. And if your review text is conveying something you don't like about the resource or the update you're reviewing, it can seem appropriate that the numbers of stars should reflects the thing you don't like. But the end result of that is you're reviewing the whole resource as 3 stars, rather than just posting a message and a rating associated with that specific message.

I think this is really a case of user error and now knowing how to use the site correctly. If you have a request or a question or something you'd like to note the creator about, just post it in the discussion. If you want to review the resource itself, review the resource itself, don't just slap a rating to reflect a specific gripe you have about the resource.

 

[everlaster]

One useful statistic that might be trackable is: Number of times included as dependency on Hub.

Sorting by that would likely raise the cream to the top of the results.


The Hub tracking and sending out a list of everything you've downloaded each month with a reminder (and quick links) to review/rate would make a big difference as well I guess.

This is a great idea for specific types of resources that are often used as assets. It wouldn't work for scenes though. But for clothing, hair, textures etc. it'd be great - for a resource meant to be used as a dependency (in addition to personal use), the number of references is arguably a more important metric of popularity than the number of downloads. A bit like for a scientific paper, the number of citations is more important than how many people have read the paper.

 

[Guy]

I think the reviews would be much more meaningful if they were not mandatory. This would also remove a lot of random comments that aren't productive feedback.

 

[Guy]

or this shit, look at this douchebag named @D91d
View attachment 136095

This dude shared a wonderful resource, made 17 updates to it (!), shared it always for free, and has to deal with morons like @D91d because of having the impertinence of making you do an extra click and consider supporting him make more stuff in case you like his assets. What's he supposed to do? Delete patreon and just slave away assets for vam hub stars?

Let us start banning people like that. If they don't like patreon links, they can skip the items that don't have the green Hub-Hosted. Or maybe make the label bigger for @D91d to see it more clearly before he wastes his click. Imo stuff like this shouldn't be tolerated

D91d makes a valid user experience point. Supporting a bunch of patreon developers and manually installing everything, along with updates, etc. is really only viable for the most enthusiastic user. The Hub could provide a store that would provide a better end-user experience that would pass on rewards to the devleopers. Not mandatory, but an app store option would be very popular. Combined with reviews and stars, you're basically replicating Amazon which nearly everyone uses.

 

[Guy]

You know, I think the stars don't reflect the quality of the content. It would be great if, when uploading content through the internal VaM interface, it was suggested to anonymously evaluate the quality according to several criteria. For example, the evaluation window at the next VAM startup. And to avoid cheating, you can get hardware ID for uniqueness

You could have two review #'s. A completely unmoderated "masses" star ranking (without mandatory reviews), and then an "expert collective" review rating, like movies do. Then people can decide for themselves what is more helpful.

 

[Assman.]

or this shit, look at this douchebag named @D91d

This dude shared a wonderful resource, made 17 updates to it (!), shared it always for free, and has to deal with morons like @D91d because of having the impertinence of making you do an extra click and consider supporting him make more stuff in case you like his assets. What's he supposed to do? Delete patreon and just slave away assets for vam hub stars?

Let us start banning people like that. If they don't like patreon links, they can skip the items that don't have the green Hub-Hosted. Or maybe make the label bigger for @D91d to see it more clearly before he wastes his click. Imo stuff like this shouldn't be tolerated

Bro c'mon, don't tell me you agree that it's ok for him to state those kind of philosophical opinions in the ratings of one dude's free assets. There's a VAM suggestions section on the forum where he can state any kind of opinions, suggestions and complaints. He can complain to the vam developers and moderators for allowing patreon links. He can go to the discussion board of the addon and complain there if he has something against that dude in particular. He can DM him and tell him he should stop hoping to get a pizza every now and then out of his work and just slave away for the passion of it man! Doing stuff like that in the ratings serves no other purpose than annoy the creator and discourage him from sharing free stuff, discourage others too, it's just petty and toxic.

Also, there's a green tag "Hub-Hosted VAR" specifically made for people that don't want to do extra clicks for free stuff. If you click on something that doesn't have it, you know what to expect: external link. It's just how the hub works, learn to use it, don't ask others to comply to your arbitrary rules lol. Is he going around complaining about everyone with external-links, or just picked on that one dude? In both cases that's a ? move

Damn, someone has anger issues

Explain to me how exiting VAM, opening a web browser, go to the hub webpage, searching up the resource, clicking the link that directs you to patreon, download the resource, drag it into the VAM folder and then rebooting VAM is an "extra click"

Now imagine having to do that for multiple resources.

You're being very overdramatic, and talking about creators "slaving away" over something so small as a review

You seem to have quite a lot of assumptions that you've pulled out of thin air to try and make a point, or is it just bad reading comprehension?

I don't have anything against patreon links, and I also don't have anything against Korisaki.

I have "something against" putting it as the only download link. If everybody with a Patreon did this, then the in-game HUB would be more or less a pointless feature.

Posting it in the suggestion section would probably not do much, as the people who do this most likely won't even look there in the first place.
Why should I complain to the VAM devs/mods about patreon links being allowed when that's not even the issue?

A lot of creators have a "Support me" button below the Download button, so that people who want to support the creator can follow that link and do so, while the rest can simply download the resource directly without having to visit another webpage to do so. Seems to work just fine for them? They still get support from those who feel like supporting them.



From what I can see, you're the only person who got super upset over this, not to mention you talk about me being toxic when you're the one who has thrown multiple insults at someone out of nowhere

Leaving a review with criticism = Ban
Multiple insults = Ok

Good logic there ?

 

[Assman.]

If there was a point in your post, it was certainly drowned in your hissyfit of a post full of insults and what not.

If you don't like external link addons in vam, that's an issue you should take with VAM through the suggestion forum

You already said this, and I even answered you on that.

And guess what? I made a thread about this in March 2021!

Question - Download buttons leading to Patreon.

I have noticed that a certain group of authors always put the download for their resource as a link that directs you to their Patreon to download the content there, instead of a direct download here on the hub. Doesn't this kind of defeat the whole purpose of the hub? Specifically the "in-game"...

hub.virtamate.com

And like I said in my previous post, it lead absolutely nowhere.

In case you have a hard time with reading, the review literally starts with a compliment for the asset..

Korisaki didn't seem to take much offence, but now about 3 months later you come with the overreaction of the year over something that has nothing to do with you.

NOT REVIEW: i'm mad right cause I was edging juggling addons but then I came across yours and I had to take the VR headset off and almost tripped with my pants off while alt-tabbing and dragging folders and then i had to do a reboot also and that gave me blue balls. Imagine everyone did this to me. I'ma set an example out of you!

Not even sure what this word salad is supposed to mean. I also don't use VR, so this must be a personal experience of yours?

I think you need to work on yourself and your issues rather than throwing temper tantrums at random people over something that was done several months ago. Nobody will take you seriously or side with you with the way you are behaving.

And please no more super dramatic stories about these creators supposedly struggling to make ends meet and no more pizzas or whatever because of one random review that you somehow interpreted into something completely different than what it was about.

 

[Toonen1988]

First thing I would to point out is, keep it civil and don't let it escalate. I know both of your frustration. This topic is something serious and it can and will affect certain people. But this isn't a exclusive Virt a Mate hub issue, but a general problem on the internet.

First of all, I do get what @D91d is saying. It's great if assets, plugins or what ever are being shared through the in-game hub. It's easy to use, you only have to press download and your done.

But for me personally, I don't mind how the content is being shared. I do use the in-game hub, but I also use external links and put it in the right map. I'm fine with either way. But I do understand that some people only want to use the in-game hub, due the friendly interface and they don't have to do much.

I also have to agree with @SPQR aswell, the review section isn't about how the content is being distributed. You can simply send the creator a private message with the question why the content isn't being shared through the in-game hub. Negative ratings like this can affect the creator itself, and soon or later they will quit, which happened already.

So now back to topic.

The problem I have nowadays with people is, if someone doesn't like something, they expect everyone else to dislike the same. We all have our own taste and preferences. Thank god for that, otherwise the world would be boring.

I don't like for example futa. I don't like it and it's not my thing, same with "monster" girls. I don't judge anyone for their kinks or their preferences but it's not my thing. But that doesn't mean I should leave "bad" reviews everywhere, because I simply doesn't like it. No, I will stay away from it and leave the people be who do like it. I'm not going to judge anyone or leave bad reviews because it isn't my thing. No....

A review should only be about the content itself. Not directly aimed at the creator nor how it is distributed. And even if you download content but end up disliking it, you can be honest and say something like "Thank you for making this content, but for me personally I don't like it, because......" and then still give a good rating for effort or the quality of the product.

If I download a clothing piece, but personally don't like it but it has good textures, good physics etc. I would still give it a high rating.

But for me personally, I don't give reviews. Maybe I should do it, to support some creators. Because I hate seeing creators leave, because they don't feel appreciated for what they do.

And my opinion is that every creator should decide if their content should be free or not. As long it isn't in conflict with any copyright.

And keep in mind, always bring arguments in a review. If it is a negative or a positive rating, give some useful information why you gave it a certain rating. Negative feedback can also be very valuable if it is substantiated. And it should only be about the content itself, nothing else. Especially not directly aimed at the creator nor how the content is distributed. If you don't agree with something that hasn't to do with the content itself, send a private message to the creator and explain your issue in a civil way.

Please dont feel addressed but we life in a childish world. Many people need to grow up and think before they do something. I can see it everywhere, not only on VaM, but also on Steam, social media and other places, even on my work. Is it in our nature to make other lives miserable, is that why we misuse certain things? We do to much without thinking and it affects other people.

 

[Juno]

Frankly if I got a bad review that was nothing to do with the actual release I would simply report the review as not relevant to the resource and get the mods to kill it and no drama, sure it hurts the ego but this is the internet boys and girls! Then I'd dm the reviewer asking them to enter the discussion section if they wished to engage regarding their issues.

But for me personally, I don't give reviews. Maybe I should do it, to support some creators. Because I hate seeing creators leave, because they don't feel appreciated for what they do.

You are really right that most creators really don't feel appreciated at all and its so dammed fast and simple to change all that yet still far less than 1% bothers to leave a like and review. Please start to give reviews and ratings as I have seen far too many people fed up and stop sharing for this exact reason and a lot of them at the start of their career and only after one or 2 posts.

The joke is that all I hear around here is people complaining about paid content being too prevalent and every days theres new paid creators going straight to paid and not releasing any free resources lazy this that yada yada. Just look at the support people get in ratings and reviews...most will prefer to take money instead of good will since the good will is so thin on the ground.

Il give you some insight, when I create a look I often spend 3 or 4 hours in daz/blender preparing face/eyes/nose/mouth morphs and exporting them to vam before then a few more hours in vam to get to a point where a models head is 90%, maybe a few more hours making morphs to change the things existing sliders cant, an hour or 2 on make up and other textures which can be many more hours or even days of texturing if I wanted to do something more ambitious like costume makeup. A few more hours to make the body with the potential back and forth in blender. Then I have to find hair/clothes to use that suits the models theme , adjust things like teeth so they don't look super janky when animated and all that is before I take the screenshots and make the hub post which can be days more work as I'm real ocd about these things. Easily 12 15 hours of my time minimum per model which I give for free and whilst I appreciate I get more likes/reviews than the vast majority of creators its still less than 0.1% and I am one of the best supported here. My girl Naiomi has at the time of writing 35 reviews out of 30,000+ downloads and she's the best reviewed OC look on the hub...

Is it any wonder that many creators think "You know what, no ones gonna appreciate me giving up all my time and effort for free, sod it I'll get paid instead"

Myself I've been told so many times to charge for my work as I would have a very well supported patreon and I'm stupid not to take your money etc but I do this cos I love the community we built here and I want to see my work used, the biggest compliment I can get around here is that someone made some pretty screenshots or used my girls for a scene and thats my motivation! Trust me I really could do with the money

The reality is I don't get fan mail or simpy people stroking my ego at all and the only ones that do rarely contact me have seen something I posted in the discord/hub media and want the model for free, yet they never liked reviewed anything I have released previously and even if I'm feeling generous and give them the model they ghost and are never heard of again. It sucks so bad and demotivates me but this is the norm around here and many other Creators report the exact same issues.

I hope anyone reading this will be inspired to get out there and leave some reviews and ratings for the content you enjoyed, leave a message in the discussions or dm the creators, engage with us! I know it sounds all cheap and petty to be hung up on this stuff but its the only way to give back since many of us don't want your money!

 

[Toonen1988]

@Juno I fully agree, it doesn't always have to be about the money. But yet, you spend much time in it.

I'm using VaM for like 2 years now and my characters still changes once in a while. Because i'm not happy enough. But I never tried external programs to be honest. So, I also spend many hours on my characters. But I never planned in releasing them.

But anyway, you could also give away free content and make use of "donations". So, your content isn't being locked behind a paywall and people who still want to support you with money, can still send you something.

You have now a webpage like "buy me a coffee." Easy to set up and easy to use. You don't force people to spend money, but you give people a chance to support you in different ways.

So, your content isn't behind a paywall and you give people a "option" in how they want to support you.

I don't mind in supporting a creator with momey, aslong its reasonable. I supported a lot of clothing creators, VL_13, Mr.Cadillacv8, AnythingFashionVR and many other creators. They make good quality clothing and that deserves my money. But they also doesn't ask insane high prices and I can live with one less sandwhich, which is probably even more expensive.