Should we allow list questions taged as **big-list** for now?

Question

As far as I understand it, the policy regarding list questions is an ongoing discussion (see here and here). The idea is that we should built up a community first, and then discuss how to handle this issue. I agree with this, but I also think that there should be guidelines for the transition phase. Right now the policy seems inconsistent.

My personal opinion is that we should handle list questions as liberal as possible for now.

I had a brief look at mathoverflow, math.SE and cstheory.SE (see for example here). Here it seems okay to have list questions at some extend. Indeed, list questions receive usually a high number of upvotes.

I would like to propose the following: (I extended the following items, some of the comments might refer to the frist version of the question.)

1. List questions are allowed, if they are cs-related, and the question is a real question. In particular, they should follow these guidelines.
2. All list questions are tagged. This might be a big-list or subjective tag.
3. All list questions should be community wikis. (Not so sure about this.)
4. If a list question turns into a open discussion or a long list of items it will be closed.
5. If the community has grown, we review our policy.

Discussions are welcome.

Answer 1

We can take a step back and try to look at the question of "list questions" a little more broadly. What is the difference between a "list question" and a non-"list question"?

First off, it's not that there are multiple good/correct/up-voted answers, and it's not that older answers accumulate more up-votes. That's all true (at least potentially, in theory) of any non-trivial question. By itself, it is not enough to distinguish "list questions" from non-"list questions".

Second, it's not that list questions are inherently (bad) subjective. Non-"list questions" can be (bad) subjective, and so can "list questions"; "list questions" can also be objective or (good) subjective.

I feel that that where "list questions" most reliably fail is in the interplay between these two factors. Either the question is too broad or hard to understand, or the answers are (bad) subjective - in the sense that the answerer doesn't bother to explain and motivate answers properly, since this is somehow not expected of answers to "list questions". The former problem is already handled by other means, and the latter can be handled by rules already governing the quality of answers, so long as we make sure to change (and enforce) expectations.

I propose that we allow "list questions" with no special treatment, but enforce the following rules as rigorously as we would on any non-"list question":

1. The question must ask for a well-defined, sufficiently-scoped list of items.
2. Each answer must provide objective or (good) subjective argument for the proposed list item(s)

Especially with regard to item 2: simply providing an item as the answer wouldn't cut it, and answers that are clearly (bad) subjective are either removed or converted to comments. With regard to 1, if the question isn't salvageable, it and all the answers are killed with extreme prejudice and without looking back.

Answer 2

The problem with “list questions” — questions like “What are some interesting results about this object?” and “What books are there about this subject?” — is that they do not call for answers, they call for items. The Stack Exchange platform is not well-suited for lists.

Lists don't tend to generate discussion. Usually, the outcome of a list question is:

• either only a few answers, which makes the answers as a whole woefully incomplete, so that even all the answers taken together do not make a good answer to the question;
• or many answers, sorted in a way that doesn't cater to relevance, with no indication of why one would prefer this or that answer, with many duplicates, etc.

Voting on items tends to privilege whatever was posted first, and secondly the most popular items. It practically never highlights the best items (whatever best can mean in context) or the most relevant ones.

All list questions should be community wikis.

Making the question community wiki doesn't change anything to the quality of the answers. On this topic, please read the future of community wiki.

List questions sometimes work on Math Overflow and CSTheory because there's a major entry barrier: only a select few like-minded people participate on the site. This significantly reduces the bad, poorly sourced content. The lists of items might not be complete or rated but at least most of the items tend to be relevant. Experience shows that this isn't the case on less selective sites.

Math.SE might be able to afford a few duds now and then, but CS.SE doesn't have the established user base and traffic volume to accommodate a bunch of low-quality questions.

My personal opinion is that we should handle list questions as liberal as possible for now.

Ouch, no. The proper response to expected train wrecks is not to watch them accummulate and cordon off the perimeter and send excuses to the families when they happen. Rather, we should prevent the train wrecks in the first place.

If the community has grown, we discuss this question again.

No, on the contrary, quality can only go down. The time to eliminate bad questions is now, not after the site has failed because there were too few actual questions amid a sea of lists.

Questions on Stack Exchange must be answerable. That means that there is the expectation that one answer will be a sufficient resource on the question. If every answer has one item, that doesn't happen. For example: do not ask “what is the best book about X?” or “I want to read a book about X, what are the possibilities?”, because they call for items. On the other hand, a question like “What collection of books and articles would make a good background for a semester course about X?” can be a good question; different answers might give different reading lists, sometimes mentioning the same book but in a different set and with different rationales. Then each answer is useful on its own.

In a nutshell: don't ask for a book, ask for a curriculum.

Answer 3

The problem with list questions is less with the questions themselves, but with the answers they encourage. We have little effective means to prevent bad answers (between downvoting as mild measure and outright deletion as hammer, there is nothing). Therefore it is easier to close questions that promote (maybe only) bad answers than to try and make the community downvote bad answers. Note that we have been allowing reference-request questions -- often inherently list questions! -- which are reasonably scoped.

So, the real question here is:

What to do with questions that explicitly or implicitly ask for flat lists of items?

First, why are lists of items a problem? Gilles has already outlined why they don't work well with the SE platform: usually all answers are of similar objetive value, ranking them makes no sense. As a collection of knowledge, wikis (for instance tag wikis) work better. Furthermore, such lists usually become -- once they converge towards completeness -- impossible to read, because there is no structure.

Is it possible to overcome these issues? Sure. What papers should everyone read? on cstheory.SE asks for semi-objective recommendations and many (of the top) answers give reasons for why you should read the respective paper. What books should everyone read? on cstheory.SE works well because there is one aggregated answer per category, thus the list is structured¹.

Consider on the other hand Beautiful results in TCS on cstheory.SE; this is a completely unstructured, purely subjective blobb of things. Look at Hidden Features of Java which is nice to skim through for entertainment, but a useless reference due to redundancy and lack of structure. What should every programmer know about web development? has escalated in rambling, subjective lists of lists, often without any justification.

What do all these questions have in common? They are broad, and often ask for subjective statements². Which of them do work (as question on SE)? Those where a relatively small group of experts has come together and carefully shaped the list in a useful form.

Therefore, I think we can reasonably allow list questions, but we need a policy that governs aggregation and structuring of the answers. As mentioned in ¹, this was tried in the early days of cs.SE with little success due to lack of consensus and cooperation. I tried to formulate a policy³ but it was downvoted (without comment).

So, bottom line: I have no qualms with list questions as long as they are kept tidy. In order to do so, we need a clear policy (maybe even mandate!) to do so. So, no list questions until we have a policy. I propose my sketch form back then again, although it might need some polishing if we get down to it.

---

1. I tried something similar here but the effort was rejected.
2. To an extent, that is true for any answer: even if I write a proof, the method and style I use is highly subjective. So there is no clear-cut criterion. "How do I best write down an induction proof?" is subjective, but narrow enough to produce good, comparable answers.
3. Since we have mods now, users do no longer need to delete their own answers; we can do that. In the long run, the community should maintain such questions on its own.