Official Poll - reactions vs. reviews for resources

[Acid Bubbles]

If both are kept then we just need to acknowledge that "like" can mean "hey this looks cool", so the ranking value is more than a "view" but less than a "download", where a "rating" (be it a simple endorsement system, which I personally like, or a star system) is worth much more. But this ends up being a fairly complex algorith

Here are a few ideas (throwing them out there):

- Only keep "thumbs up", other reactions are not really useful imho. That would help clarify things rather than giving different values to emotions (e.g. laughing is good or not? Does wow means nice or just surprised to see this?)
- "Most liked" would be one filtering, while "Best rated" would be another.
- If you're going with a "hot" section eventually, I think likes + downloads could count a lot, but low star ratings should bring that content down quickly (e.g. nice screenshot but broken scene)

 

[meshedvr]

I personally like having different reactions available.

The trending addon I'm looking at appears to allow complex criteria for determining what is "hot", and allows setting up multiple filters for looking at resources from different perspectives.

[TH] Trending

Trending Showcase your most active and relevant content so your users can find current information and get involved fast. Trending is intricately calculated by several factors based on the content type. This will aid in increasing user...

xenforo.com

 

[Acid Bubbles]

Then another idea: Keep the "like", but maybe change the other reactions? e.g. "Broken", "Dislke" or whatever. (I'm dropping this now though, this is probably not worth pushing any more, having "sad" reactions to resources is weird to me but not an actual problem)

 

[meshedvr]

Then another idea: Keep the "like", but maybe change the other reactions? e.g. "Broken", "Dislke" or whatever. (I'm dropping this now though, this is probably not worth pushing any more, having "sad" reactions to resources is weird to me but not an actual problem)

The problem is the reactions are for the whole site including forums. Resources can't have their own reactions types. So I am just using the defaults Xenforo came with.

 

[MacGruber]

@meshedvr I just tried to sort resources by "Rating" in descending order. For some reason my own resources (PostMagic+Essentials) appear twice. Best guess is that the custom comparison function you implemented (combing review stars with reactions) is not entirely reliable, causing items in the list to be duplicated. Maybe that can happen when A<B and B<A?

 

[meshedvr]

@meshedvr I just tried to sort resources by "Rating" in descending order. For some reason my own resources (PostMagic+Essentials) appear twice. Best guess is that the custom comparison function you implemented (combing review stars with reactions) is not entirely reliable, causing items in the list to be duplicated. Maybe that can happen when A<B and B<A?

I don't have custom combination sorting released yet. What you see there with "Rating" is what is built in. I will take a look, but I have not seen that issue.

 

[meshedvr]

Strange - I see your resources twice as well. It seems to only be you. What did you do?!?!

 

[MacGruber]

It seems to only be you. What did you do?!?!

No idea

 

[meshedvr]

No idea

Ah - it seems to be some issue with the endless scroll addon I'm using and filters. It looks like when you set a filter after some content getting paged in it gets messed up. I'll see if I can sort it out.

 

[meshedvr]

I turned off the unlimited scroll addon and it is still an issue. You can see for yourself if you go to 1st page:

Free

hub.virtamate.com

Then click page 2 it repeats some of the items. I'll see if I can fix it.

 

[meshedvr]

@MacGruber I have the duplicate resources issue "fixed". It seems to be some kind of bug that happens when resources per page is set below 8. Most users would have this set much higher, so they probably never hit this issue. But I was using an on-demand loading endless scroll addon and I had it set to 5 resources per page. This was mainly to test that the page-loading was working. Now that I know it is working I have changed this to 10 and the duplicates are gone. It was actually hiding top-rated resources and replaceing them with duplicates so this was an important fix. Thanks for pointing it out.

 

[DJ]

For an in-game rating system, perhaps a brief questionnaire? Does the item do what it's supposed to? Is it broken in some way? Is it hard to use? For clothing, is it simmed well? Is it tagged well?

people aren't going to write reviews in game (at least not in VR), but feedback about what is done well and what is not would be helpful both for users and the creator. I notice many clothing creators don't tag anything, which is annoying, and the issue may be that they just don't know about the system. A feedback system that educates users and creators alike about what makes an item high quality could serve to improve resource quality across the board.

 

[meshedvr]

For an in-game rating system, perhaps a brief questionnaire? Does the item do what it's supposed to? Is it broken in some way? Is it hard to use? For clothing, is it simmed well? Is it tagged well?

people aren't going to write reviews in game (at least not in VR), but feedback about what is done well and what is not would be helpful both for users and the creator. I notice many clothing creators don't tag anything, which is annoying, and the issue may be that they just don't know about the system. A feedback system that educates users and creators alike about what makes an item high quality could serve to improve resource quality across the board.

Yeah I think that would be really nice if you could review something in-game with a few simple clicks. Star-rating and maybe a bunch of checkboxes to add canned comments and a space to add free comment. VaM knows what package things came from, and if user opted to login to hub it should be pretty simple to show this option to review or revise existing review.

 

[Acid Bubbles]

A simpler option would be to simply ask what's the main reason for bad reviews with a few choices, and nothing for 4+ stars. Something like "what could be improved" instead of "what's the problem could also help with general attitude towards feedback. Something like I wasn't able to make it work, The plug-in crashed too often or this wasn't useful to me.

Also about alpha version of plugins, this could destroy future ratings. I'm thinking of two options. Either give the option to reset the rating once when moving out of alpha, or simply ask in the rating window to rate potential. Those are just thoughts I'm not sure this is strictly necessary, but it'll certainly affect how comfortable people are releasing prototypes and ideas. And in my experience, prototypes quickly become dependencies regardless of how many warnings you give.

 

[meshedvr]

Each version of resource has it's own rating, and I believe Xenforo weights this somehow in what is displayed, possibly favoring the rating of newer versions over older version. I haven't looked at the details, but I'll check it out. In any case, it could be changed such that older version ratings don't matter that much.

 

[meshedvr]

@Acid Bubbles Yeah so I just tested with this https://hub.virtamate.com/resources/test.120/ and it clearly shows the overall rating based on version 9001: https://hub.virtamate.com/resources/test.120/history

I posted a new update and submitted a 3-star review and the overall still shows the last version. I don't know at what point it would start showing the new version ratings. But this seems like it would encourage updates to improve the rating, and older ratings don't count unless someone looks at the history tab. This could even be a positive because it could show the improvement over time.

 

[MacGruber]

Wouldn't that mean I loose all the ratings when posting a minor update?

 

[meshedvr]

Wouldn't that mean I loose all the ratings when posting a minor update?

Yes it does seem that way. Perhaps multiple ratings could be shown. All versions weighted + Latest version

 

[Acid Bubbles]

Yeah it's hard to build something that doesn't allow to game the system. I'm posting very frequent updates, this shouldn't reduce the value of previous ratings, and you don't want someone to spam updates to cancel bad ratings either (especially if they have a patreon, there will be incentives to maximize rating values for financial reasons).

I think the Steam model is interesting. You average the last N ratings and that's the value you care about, regardless of versions. Also having the "overall" rating is interesting, since it shows whether the plugin "evolved" or "broke".

 

[meshedvr]

I dug further into this. The test resource has

Version 1: 1 3 star
Version 9000: 1 2 star
Version 9001: 3 1 star 1 2 star 1 5 star
Version 9002: 1 3 star 1 5 star

Overall has:
rating_count: 5
rating_sum: 16
rating_avg: 3.2
rating_weighted: 3.067777
review_count: 9

So I think what is happening is users have updated their rating over time so there is still only 5 total, but the history shows all ratings (9 ratings from 5 different users).

So users can change their rating on a new version and only the latest counts in overall average. But all ratings count in the average.

This is probably what we want, but it doesn't solve the problem of alphas and low ratings. Perhaps alphas should be released as separate resources. I think it would require making a fairly complex addon to support alphas with ratings disabled or not count after official version is launched.

 

[DJ]

Yeah it's hard to build something that doesn't allow to game the system.

Speaking of which, is there any mechanism to prevent me making a dozen accounts to write glowing reviews of my POS paid content?

 

[meshedvr]

Speaking of which, is there any mechanism to prevent me making a dozen accounts to write glowing reviews of my POS paid content?

It's possible, but it would be a pain for you to do in a way that would not be detectable. I don't care to reveal how exactly to do that.

 

[fossilcloud]

originally facebook came up with the emoji reactions due to the criticism that they force a yea-sayer commuity because they only allowed upvotes. however that didnt make it any better. thumbs up, love and laugh often mean exactly the same and positive reactions are not counted against negative ones. star ratings however are just one dimensional. a real and actually quite simple solution would be to have multiple ratings for different quailites like ebay rates sellers by speed, service and product quality.

just have ratings for static visuals, interactivity, technical execution and a combined score. instead of 4 rows of star ratings each rating could have different symbols.

 

[Switch]

This would be best as a ranked choice vote if possible. That's a better way to discern between more than 2 options. Although at this point in time the preference is clear.

 

[Maxman3D]

I like the star/review system since it's what I am used to at other places and like the idea of posting an actual review when I feel the addon warrants it.