Talk: Real Vetting/Archive 6

Criteria

There's now a draft and a separate talk page for discussing the criteria. Please use it! --Hhhippo 21:39, 4 June 2007 (CEST)

Loose ends from archived discussions concerning criteria:

Hhhippo suggests: Should the "Strength" criteria be split into something like "Power" and "Usability"? Not so experienced players might look for a build that they can handle, and accept that it has limited power, while others don't care how difficult a build is to use (or to get the equipment) but want maximum power for a difficult task.
Shireen suggests: Idea should be changed to Concept, as it's a stronger word to reflect the attributes meaning. You may want to change out Strength with 'Overall' or another neutral evaluation word, as strength alludes to damage output, and seeing low damage output may skew voters perception.
The following threads have been moved here as sub-sections: --Hhhippo 21:23, 3 June 2007 (CEST)

Improvement suggestion on Score calculation and voting categories

Any build can be described by the following voting categories:

effective
innovative
adaptive
universal
easy to use

Effective is simply a word that describes better what Strength is in the original version.
Innovative is carried over from the original version.
An Adaptive build would be a "paper" build that stands a chance against "scissors". If you are a ZB bonder, you can always leave the bonds when facing enchantment stripping, and just use ZB and whatever other spells you have. If you are a warrior, you can always use an interrupt or a knockdown to stop that anti-melee necro.
A Universal build is one that can be used in more than one location for the same purpose. I have deliberately changed potential into these two voting categories, because these describe better what potential is meant to be, and this way they can be weighted differently in the total score.
A build that is easy to use (couldn't think of a better name) is obviously easy to use. People want easy builds, so why not give them this voting category.

Now, these categories have to be weighed.

A good build has 65% total score.
A build that gets the maximum score on both Effectiveness and Ease is a good build.
A build that gets the maximum score on both Effectiveness and Adaptive is a good build.
A build that lacks innovation, universality or ease can still be great.

So Effectiveness, Adaptive and ease are the core scores.

The other two are less important; the innovative part is hardly interesting for a total score.

I suggest these categories are weighed at 50%, 5%, 20%, 10% and 15%

Ifer - 82.75.190.21 09:47, 22 May 2007 (CEST)

Err, more like a build's performance should count for at least 80% of the build's rating. I would prefer 100%. -

Krowman {{sysop}} 02:06, 26 May 2007 (CEST)

No. Here's why. Say I have an Assassin boss farming build that can kill the target within a second, but takes ages to get right, is not useful versus any other boss and can't deal with bad mob spawns. That build would get the maximum score in effectiveness, but can hardly be called a good build. -- Ifer 82.75.190.21 00:12, 27 May 2007 (CEST)

Uhh, and what exactly is wrong with that? You can't expect to instantly, or even quickly, master any build. Builds should be specific to their environment. If you want to farm a certain boss, you fill your bar with skills that can get you to him, defeat him, and survive his mob. Those specific skills will vary depending on him, his mob and the critters between you and his spawn. If the build can successfully farm that boss, then yeah, it would be an effective build. Srsly, under your logic, any farming build or team that is set to farm one area (I'm thinking DoA, the old Urgoz, the Deep) would get a lower score because it can only work in that specific location. Who cares? If it can farm that location well, it is a good build. Btw, I seriously doubt your hypothetical build could ever be realistic, and therefore has very little value as a demonstration of why my argument phails. -

Krowman {{sysop}} 06:38, 27 May 2007 (CEST)

Great build is not the one that can farm W boss's and run away. Great build are the one that can be used every where and still work 100%. The whole idea with this policy is to split lame W solo builds from builds that is Great. gcardinal 15:05, 4 June 2007 (CEST)

RA builds =/= GvG builds =/= PvE builds. No build can be used everywhere. If a build like that ever came into existence, then we could justify sorting builds under that criteria. Until that point, having a build's Innovation score affect its rating will just give viable builds a lower rating, with no real benefit. Having the Innovation score displayed alongside the Strength or w/e score, and having it as a searchable criteria, would be great, but not having it influence the build's overall rating. Imagine what would happen with common, tried-and-true builds under this system. Shock Axes would get low scores, because they have all the staples of good Warrior builds. 55-style farming builds' scores would drop, because the idea is hardly original. Good Monks would lose points because they have hex and condition removal components, maybe some e-mgmt, survivability, etc. These builds would all suffer, while builds like this one would succeed because it may be the best use of a bad elite. People use certain skills for a reason: they are good. Similarily, people don't use other skills because they are bad. A build using these unpopular, bad skills should not be promoted on the wiki over solid builds like the ones listed above. The only positive aspect a set of criteria such as these would provide would be to build authors, who could take some sort of pride from the fact that they made a highly-rated Fox's Promise build. It would not help anyone who came to the wiki looking for advice; it would only make their search more difficult. -

Krowman {{sysop}} 14:09, 18 June 2007 (EDT)

Naming of Categories

The nomenclature for the criteria should look as follows:

Strength should be replaced by Effectiveness
Potential should be replaced by Universality
Innovation should be replaced by Ingeniousness

That should help clarify the meaning of each, and it makes them look more uniform. The various standards for 1-5 ratings for effectiveness can be found below. In the cases of Universality and Ingeniousness, the ratings are more straightforward and we felt descriptions weren't necessary for each level. *Defiant Elements* +talk 07:26, 1 June 2007 (CEST)

Sorry my bad english... but maybe you can find another word for Ingeniousness... I had to read it 3 times before I got it. Can be hard to non-english users. gcardinal 14:53, 4 June 2007 (CEST)

Description of Numerical Values

5 = This build excels amongst its contemporaries. It is well-received by the Guild Wars community, and worthy of PvX Build's highest rating.
4 = A build whose qualities demonstrates better than average capabilities in a realistic environment.
3 = A build which has all necessary prerequisites in order to achieve the minimum level of effectiveness required to qualify as an acceptable build.
2 = This build does not function adequately, either because it features an innefective concept or lacks the necessary elements of a better build.
1 = An ineffective build as defined by PvX:WELL.

- Krowman {{sysop}} 08:01, 1 June 2007 (CEST)

Category names

Doesn't it sound strange to put candidates for deletion into a category called 'Archive'? Shouldn't we skip the two-stage structure and just call the categories Great, Good, Archive and Trash? --Hhhippo 20:45, 22 May 2007 (CEST)

On a side note, I agree with Hhhippo, it would be better if the categories were renamed to simply "Great, Good, Store, Trash"... or possibly replace "Store" with "Archive". To denote builds as "working/great" would imply that there could be "not working/great" builds as well. When a build falls out of grace for whatever reason, like skill changes.. then it would go to "store" or "archive" where I think it should show what the previous score was for the build and why it doesn't work as well/at all anymore. "Great" for builds that currently work very effectively (based on score), "Good" for builds that work fairly well, may not be very user friendly.. or very universally effective, but still good, "Archive" (or store, whatever) for builds that had previously been favored but have fallen out of grace for whatever reason (had once been successfully vetted), and "Trash" for builds that did not pass the vetting process. But if you wanna still use the 0-5 scale, I would suggest just labeling them "Great, Good, Ok, Bad" or something like that.. store/trash are essentially mislabeled if it's just based on average vote. -- Spore 20:20, 30 May 2007 (CEST)

Some more points about categories:

What's the name of the category for builds that are not yet ready for testing/vetting? The policy says "Drafts", currently we have "Build Stubs" for essentially the same purpose.
How do we indicate that testing can begin? Move from "Build Stubs" to "Testing" as before? And does that automatically trigger the rating tab? Will "Testing" have the known usage subcategories (PvP, PvE,...)?
Once there are enough votes, builds will automatically be moved to one of the 4 quality categories by the extension. Do we have consensus about the names of these categories being "Great", "Good", "Archive", "Trash"? Any opinion on replacing "Archive" by "Store"?
How do the usage categories come into play here? As subcategories to each of the quality categories? Or do we forget about the wiki category concept here and leave the filtering to the new search engine?

--Hhhippo 19:09, 5 June 2007 (CEST)

Build modification after voting

What if, based on poor ratings, the author makes significant changes to his build that should result in higher ratings? Will the original ratings still count against him?

What if an author creates a cookie-cutter build to garner initial high ratings and then changes his build dramatically to be something esoteric/creative, so that the high votes give his weird build exposure it would not normally get?

My point is, while I have nothing against this vetting system, why the hell are you implementing it on a wiki? Why not just write your own simple CMS based around this concept? Wiki and this system are diametrically opposed.

Tanaric 03:07, 1 June 2007 (CEST)

Significant changes (good or bad) could be the basis for a re-vote, but has that ever been much of a problem? Over at Gwiki, if someone changes and breaks a good build, someone else would come along and fix it. The central ideal behind a wiki is that people are generally good, not bad. Look at everything people have been able to create with a wiki. Yes, there are some vandals and troublemakers, but that's what we have administrators for. If someone is motivated to get their build vetted, they are not going to be constantly changing it, and constantly having it sent back to Untested. In theory, it could be a problem, but in practise, it never really has been. -

Krowman {{sysop}} 04:15, 1 June 2007 (CEST)

No, you misunderstand my concern. Under this system, a build seems to be "owned" by its author. He has every right, then, to change it later as he sees fit.

Beyond this, it doesn't matter that people are good. If being good means not significantly changing your build after its posted, and if being a vandal means altering somebody else's build, why use a wiki in the first place, where significantly changing your content and editing others' work is the name of the game?

Tanaric 05:07, 1 June 2007 (CEST)

I don't understand what problem you see in it, when a build needs to be changed of people want it to change they discuss on the talk page. It happened at guild wiki and will happen here. Discussion is just where changes should be and if they turn out to be accepted by people then it will be changed, if the changes are dramatic it would be thrown into a re vote. --

05:29, 1 June 2007 (CEST)

Hmmmm... I think I see your point Tanaric, on the other hand, I fail to see how this system would directly cause a severe deviation in the way things are run. As Krowman points out, if a good build is broken, people fix it. If the opposite occurs and a build is significantly improved, then we re-vote. Although it will have to be edited as a result of voting within an extension, PvX:NOTIFY provides a good core structure for the specifications for a revote.

Again, perhaps I am missing something, but why does this system imply ownership. The fact that votes can be reevaluated/changed by the original voter, combined with the fact that Administrators can render old/incorrect votes null and void appears to limit the impact of such an issue. If content changes, we can still change the votes. Based on my limited grasp of the coding that goes into an extension, I see no reason why we couldn't a) Only allow the last x votes to count, or, b) allow an option so that Administrators can simply click a single button to restart the vote count.

Because of the aforementioned flexibility that can be built into the program, I fail to see a significant change from the GuildWiki system, at least as far as the problems you have brought to light so to speak.

*Defiant Elements* +talk 05:31, 1 June 2007 (CEST)

Alright, I'll start this thread over, because people are only seeing one side of what I posted and aren't coming to the point I need them to. :)

This is a pretty good vetting system. However, one thing is important to it: the build article needs to be somewhat static. If a build changes significantly, the votes need to be reset. Why are you using a wiki as a backend for a system that relies upon the builds being mostly static? It seems a system in which only the original build editor can edit the article makes more since, with perhaps a comment system for limited collaboration. Such a system could be implemented in your language of choice far easier than a wiki extension. I just don't see the benefit of using a wiki backend here.

Tanaric 10:59, 1 June 2007 (CEST)

That is actually a very good point. I still think a wiki is a suitable backend, since there are various reasons for editing build articles. However, we need clear regulations on who may when do what kind of edits to a build, possibly as a policy of its own. Here are some suggestions:

While the build is a Draft/Stub in the Build: space, everybody may edit it. Edits should aim at improving the build itself or the article, and not turn the build into a completely different one. Consensus with the original author should be sought.
Once vetting has begun, only minor edits should be done. Minor edits are corrections to grammar or formatting, adding of variants or improvements to the build's description that don't change the build itself.
If there is a need for a major change, e.g. as a reaction to changes in the game, consensus should be reached on the talk page before doing the change.

In case of a major change to the build or a relevant change in the game, we need a policy for re-voting. Something like PvX:NOTIFY, adapted to Real Vetting. A possible solution might be putting the build back to testing and de-activating all votes given before the change (but leaving them visible). I'm not so sure if one needs the notification part, since voters should have the build on their watchlist. Btw: Will voting put the build on the watchlist automatically? Obviously, this needs to be fleshed out... --Hhhippo 19:41, 5 June 2007 (CEST)

I wouldn't bother with the auto-watchlist voting. If ths site becomes at least as active as GWiki's build section, users may be voting on dozens of builds a day. That'd clog up your watchlist pretty quickly. As well, if a user wanted to see of a list of every build that he or she has voted on, they could just as easily use the contributions page. -

Krowman {{sysop}} 00:29, 7 June 2007 (CEST)

Votes are not static. Each user can rate a build, he is free to edit it whenever he wants to, he can delete his vote and administrators can "rollback" vote - "rollback" will remove the vote with given reason. User is free to come back and re-rate build if his rating was removed. I do understand concerns about significant changes in build and how it will affect ratings. However W/Mo will stay W/Mo and will never end up R/E.

This problem is quite big not only here but on Wiki sites in general. One way to solve it are by introducing revisions. All builds will start with revision 0.1 and each time build undergoes significant changes revision of the build can be updated (by admins, build masters etc) to 0.2. Each time new revision is approved by admins/buildmasters/superusers/etc can choose level of importance on changes that was made.

Level 1 - just updates revision number
Level 2 - Some changes was done, most of the minor so only notice on the build page where it will say "Please check your votes". Users will re-view them ratings to make sure that everything as it supposes to be according to new revision.
Level 3 - some major changes was done and all current ratings call for new rating. All current ratings will be moved to "Ratings for revision 0.x" and notice to all users who originally rated this build will be sent so they can come back and rate build again.

And one again many changes can be done 1-10-100 before new revision/version is decide. It will depend totally on each build. This revision system will go well with "work-in-progress-bar" idea that I liked so much. Where work status can be changed along the way. And when it reaches "ready to rate" or something people will rate it. Anyway they are just some ideas and we are not going to implement them all right away, but I see revisions are the only way to efficiently control work flow. gcardinal 09:00, 6 June 2007 (CEST)

Revision system could work fine. -

Krowman {{sysop}} 00:29, 7 June 2007 (CEST)

voting categories

What the final on voting categories? I remember asking this question several times and still I dont see a final list with names and descriptions. Please take into consideration post from anonymous users he/she had some really good points that I support. Ones again start, discuss and provide a result in form of list with categories names and description. This is 3rd reminder so please do it or I will decide on it my self. gcardinal 14:59, 4 June 2007 (CEST)

Just to be sure: do you mean categories or criteria? (Well, we need a decision on both, anyway.) I'll post some thoughts in the (european) evening. --Hhhippo 15:18, 4 June 2007 (CEST)

Hi Hhhippo, I am not sure how to name then but its about * Idea – 15% * Potential – 15% * Strength – 70% . new names and full and complete description. They must be easy to understand and be representative. Mistakes here can lead to many problems and miss understanding here. gcardinal 15:32, 4 June 2007 (CEST)

I made a draft and a separate discussion page for that. Let's see what happens. --Hhhippo 21:39, 4 June 2007 (CEST)

ideas

I have spend some time on this extension and I got this idea. It may look like to much on the first look but I think it will make sure that people understand what they are ratings actually mean. A few ideas:

After rating is submitted user will see textual representation of his votes, total score and what kind of category build will end up based on this users vote. After that user will have to confirm and save rating or to cancel and edit.
Textual representation: if user rate 3 on Effectiveness text describing that will be shown. And on overall rating 0-20, 20-40, 40-60, 60-80 and 80-100% will also have they own text comments that will be shown to user.

All this is to make sure that user fully understands what he is voting for.

Another thing that will maybe make this system even strong: build masters. Idea introduced in some policy's. The basic of it: Vote from build master counts for 3 ratings, admin rating counts as 2 and user rating counts as 1. This will ensure that everyone will have a chance to cast they rating and as the same time it will be ratings from high ranked users to use as reference or guideline.—The preceding unsigned comment was added by Gcardinal (talk • contribs) at 09:45, 5 June 2007 (CEST).

Good ideas, I support both of them. I guess Build master and admin votes will be marked as such in the list of votes, right? --Hhhippo 12:15, 5 June 2007 (CEST)

Yes they will be marked very clearly. gcardinal 12:48, 5 June 2007 (CEST)

Why should some users get more power then others? I thought the point of the build master or moderators was to manage votes and get rid of in valid ones?--Aliri 20:06, 5 June 2007 (CEST)

And who votes these "build masters" to their position? The other "build masters" with similar opinions? - Skakid9090 20:46, 5 June 2007 (CEST)

I feel some deja-vu regarding this discussion. Should we first get everything else up and running and put the idea of higher counts for build masters and admins on hold for now? It can be discussed later, in terms of a separate policy, and if approved be included in the rating system. --Hhhippo 21:17, 5 June 2007 (CEST)

With the system that allows admins to strike out votes that don't correspond with the build it should get rid of the need to give people higher ranks, if a user gives a bad vote where they say its just good because it works they will have to be stricken out or clarify. If each voter is forced to completely explain the vote then it just about removes the experience levels and all votes would therefore be equal.--Aliri 23:16, 5 June 2007 (CEST)

Yes, the combination of the ability to strike votes and having a higher-ranked vote is a bad one. The potential for abuse is too great. While it certainly would be nice if I were able to just tell people that a certain build is good and not have to go through the vetting procedure, the downside of the idea outweighs its positive aspects. I know we wouldn't be giving the priviledge to people we don't trust, but we really don't need that kind of system here. I strongly disagree with the ranked votes idea at this stage in the wiki's development. - Krowman {{sysop}} 08:09, 6 June 2007 (CEST)

/Agree Readem (talk*pvxcontribs) 08:35, 6 June 2007 (CEST)

I nominate Jupsto as a build master! - Skakid9090 00:34, 7 June 2007 (CEST)

That's not "constructive." -

Krowman {{sysop}} 00:40, 7 June 2007 (CEST)

Yeah, I have to agree with Krowman, while I think that perhaps we may need more people to control the number of vote (i.e. more people with the ability to delete votes, etc.), Ranked Voting gets a no from me, and, at this point, I would vote to table the entire issue in favor of focusing purely on the necessary aspects of implementation. We can discuss this all later, for now, I say we drop it. *Defiant Elements* +talk 01:04, 7 June 2007 (CEST)

Sorry if this violates some policy buried away somewhere, but ranked user voting is a load of f***** b******. Favouritism in a sense, it's semi-corrupt in the way that it can easily become corrupt and it's just not even. It's not DEMOCRACY anymore (Thinking back at this, what if it's communism...?), which is basically what this wiki is about. I strongly disapprove of this idea. I bend over and present my 'le derriere' for a nice spanking. ~~ Napalm Flame ^_^ (talk)(contributions) 14:54, 10 June 2007 (CEST)

First of all ranked user voting is not going to happen as you maybe can see from comments above and it was just and idea for discussion. And second of all please don't use f-words, you can make your self perfectly clear without them. gcardinal 18:45, 10 June 2007 (CEST)

Okay, won't use them, just couldn't help it because I feel so strongly against things like that. ~~ Napalm Flame ^_^

(talk)(contributions) 19:58, 10 June 2007 (CEST)

Yeah and I feel strongly for ppl that are not around when so f**** much need help, but s... happens :) gcardinal 20:46, 10 June 2007 (CEST)

Now now, don't be hypocritical. ;) ~~ Napalm Flame ^_^

(talk)(contributions) 21:30, 10 June 2007 (CEST)

I think you actualy mixed communism with a dictatorship (or how you call that in english). Communism is a economic system in the first place. :D (does it matter? Nope) Cookieaddictedmonster

Progress

Hey, I was just wondering if there was any ETA when this could actually be put into place.

I'm sure that it is hard work to create the extensions/systems needed but in the original proposal we voted for it claimed it would be within a week...

Thanks

its been more then a week... wow, i would really like to start vetting some war builds.