What makes an answer good, and what makes an answer get feedback?

Question

I have been wondering: does the quality of an answer and its upvotes always correlate? It should, but it obviously does not. Obviously, because not everybody gets all aspects of every answer. Differences in style and taste influence voting behaviour, too. But I have the impression that there is more at work.

I have observed that those answers I put the most thought and effort into are often those with the least reaction. I am sure the effect is not restricted to my own answers, but those I have a good overview of, so please bear with me.

Let's look at some recent examples.

• How to call something that can be either a terminal or a nonterminal? -- trivial answer. 5 upvotes.
• Perplexed by Rice's theorem -- very simple, I even could have written it down better. 13 upvotes!
• Is it possible to always construct a hamiltonian path on a tournament graph by sorting? -- a basic, maybe subtle observation. 5 upvotes.
• Are supersets of non-regular languages also non-regular? -- very basic. 6 upvotes.

On the other hand, we have:

• Can truth tables be used in non-boolean algebra to derive functions? -- took me some research. It's nowhere near comprehensive or conclusive but I think it sheds some light on the issue. No votes, no comments.
• How to describe algorithms, prove and analyse them? -- Very comprehensive, imho. That thing took me hours. 5 upvotes.
• Determining Probability from a Graph -- apparently answers the question completely. 3 upvotes.
• A polynomial reduction from any NP-complete problem to bounded PCP -- Took me a while of researching and tinkering. If I may say so myself, it's quite elegant a reduction and exactly what the OP wanted. 1 upvote.

Obviously, I am in no position to judge the quality of my own answers, but I can't shed the feeling that putting in significant effort -- that is answering questions I don't immediately know the answer to -- is not worth my time if I only consider the SE reward system (I learn something, too, but that's not the point here). There are no negative comments, so I have to assume that the answers have no obvious problems. They just don't get any attention.

The latter answers partially share some common characteristics.

• The questions were not new when I answered (see also the answers of our answer meetup, few of which got significant feedback).
• The questions are sometimes harder or more obscure than average.
• The answers are long and sometimes technical.

Are these obervations sufficient to explain the voting difference? Do people not look at new answers to old questions? Do they not read long and/or involved answers? Are shorter answers always better?

Maybe my answers are bad. If so, why don't I get comments so I can improve them and do better next time?

I'd like to understand what happens here. If my worry is warranted, this community does not reward those posts we arguably should treasure the most, namely those that require thought and time. Instead we reward those that stick to their comfort zone and answer the "easy" questions¹. In other words, we do not encourage people to tackle the harder questions (which are responsible for most unanswered questions) or those that require more text to answer.

What are your thoughts? Is there a problem, or am I overreacting? If there is a problem, how can we solve it?

---

1. Note that I don't judge those that do. They make good contributions and their answers are typically high-quality because they know their topics inside out.

Answer 1

You're worried that there is no incentive to write good, difficult answers to non-trivial questions. By "incentive" here you mean reputation points. However, I think the real incentives are (1) helping people, (2) honing your technical skills. If you're here just for the popularity contest, then perhaps you should only address the low-hanging fruit.

Personally, while I enjoy the popularity-contest aspect, I think that my largest benefit from this site is the challenging questions that show up now and then. Easier questions — mostly homework questions, as far as I can tell — get mostly hints from me now. It used to be frustrating that answers that required some thought and ingenuity were not being rewarded, but nowadays I feel that you're really rewarding yourself by answering them, and reputation is just a way to get you addicted to the site.

If you find yourself regularly spending hours solving questions and writing extremely nice answers, then perhaps you shouldn't. Or perhaps you should understand that doing that, you're not only training your problem-solving skills, but also perfecting your writing skills. You are doing all of it for your own benefit, regardless of how much "official" reputation you gain.

Answer 2

This could, perhaps, be related to a well-known phenomenon. I don't remember the exact name, but the idea is simple enough. Ask a room of people what color to paint a shed, and everyone will chip in an answer; if the answer has already been suggested, it stands to reason that many people will upvote the answer they would have otherwise given. Ask the same group of people how best to interpret certain allusions in Dante in the context of medieval Christian literature, and it is likely that many fewer people will have an opinion, much less an answer, and therefore no answer - no matter how good an answer it may be - will be likely to receive as much attention.

Your trivial, easy-to-digest answers to simple questions were treated well, and your thorough, complex answers to more delicate questions were treated badly. I think that, as it is, the bounty system begins to address some of the underlying injustice inherent in all this, although imperfectly. That is, in order to receive a greater reward for a more obscure answer, you would currently wait until someone offered a bounty to compensate for your effort. I don't know whether there's any better, feasible solution. Some possibilities:

• An automated bounty system, so that questions which remain open and up-voted, without an accepted answer, begin to accumulate larger and larger bounties over time. Pro: eventually, every question will have a bounty large enough to attract the attention of experts. Con: the community would possibly need to be more vigilant in policing posts to avoid clutter and abuse.
• A system (possibly integrated with the flag system) whereby a few high-rep users could vote to award a bounty, after the fact, to a particularly deserving answer. Pro: good answers that don't receive due attention can be brought to the attention of people able to compensate the answerer. Con: increased burden on high-rep users to review answers.

I don't particularly like either of these, but I think that, ultimately, a modification to the bounty system could further help mitigate the problem you're seeing.

Answer 3

I do not see any real problem here.

The upvotes indicate (roughly) how many people have read the answer and found it useful. This is the feedback that I want to get, and this is what the system provides.

Of course few people will read long, technical answers. But I think it is nice to know whether it is 2 or 10 in a particular case.

There is no need to artificially inflate the upvote numbers. That would make the site less useful: you would then waste your time writing a long, technical answer and the system would mislead you to think that many people have read it. The current situation is fine: you get the real statistics and you can use this feedback to decide whether you want to write long answers in the future.

If a high-rep user feels sorry for someone who has written a good answer that has received too little attention, they can always award a bounty.

Answer 4

The problem is that the up/down votes on questions/answers are quite subjective. If I see something with which I'm familiar (probably homework scale) I'll be in a position to see if the answer is right or wrong, brilliant or plain dumb plodding. I'll feel confortable voting (and the answer probably took a few minutes to compose). For unfamiliar, complex questions in my area of expertise I'll try to check them out and vote on them, but naturally the number of people in position to do so is much less, so less votes either way. Most of the more intricate questions/answers I'll just have to pass over.

Some measure of "effort spent" doesn't sound too bad, but is it "effort spent" or "intelligent effort" what we want to reward here? I'd contend it's the second. And effort isn't really measurable in any sensible way either.

As the saying goes, SE measurement of post worth stinks. It just stinks less than the alternatives ;-)

Answer 5

this meta question just newly popped up after yuval's answer. will throw in a opinion here after about 1 yr of heavy use of stackexchange sites & my subjective experience & noting/thinking about reactions. think this is a very important question that cuts to the heart of stackexchange, and the modern evolving field of "collective intelligence". there is no real guidelines ever posted on stackexchange, anywhere, for voting. moreover even if it was posted, who knows whether people conform to it. this gives huge power & freedom to the audience but also there is some major mystery there.

If my worry is warranted, this community does not reward those posts we arguably should treasure the most, namely those that require thought and time. Instead we reward those that stick to their comfort zone and answer the "easy" questions.

imho this is exactly correct, but yes, its probably a feature of stackexchange voting system mixed with human psychology. yes, would like to analyze more what answers get upvotes and which get no response or downvotes. its a excellent area for further analysis. lacking any serious quantitative analysis, we have our subjective experience. would agree—mine fits in with yours.

part of the problem is that very few people can judge very difficult questions and answers. there is only a small audience for these questions/answers. in science, sometimes the most important questions have a very narrow audience. this is the "leading edge" of research which is highly abstract, technical, and which only a few can grasp or understand. so probably people are refraining from voting out of not knowing whether the answer is good or not. there are possibly very good answers to more difficult questions but very few people can realize that based on their background.

people do not seem to "reward" votes based on how much knowledge or research they think went into the answer, or how hard the question is. they seem to vote based on whether they think its a "good" answer ie "correct". generally this also requires it be "succinct". it takes too much mental effort to evaluate complex answers. yes, stackexchange is oriented around snap answers. the basic voting system pushes it in that direction. its part of the architecture.

another point is that I suspect a lot of voting is oriented around peoples offline reputations or background. people who participate under pseudonyms are possibly at a disadvantage to "real people". also people who are elite in a field probably easily get more votes for their participation with less resistance or pause from the audience. this is not nec. a bad thing, we probably want to encourage "high rep" people (judge by non stackexchange "real reputation/credibility") to participate the most.

basically, stackexchange is only one possible architecture for collective intelligence systems. its relatively young-- less than 5yrs old. there are some other experiments, but the whole field is very young, its not much older than "Web 2.0" technology. if people want different emphasis/focus [which I myself strongly feel at times] then other directions will have to be explored by new pioneers.

one idea I had once was a separate voting bucket or axis where people could try to estimate how "hard" a question is. an answer on a very hard question might be valued more. or something like that.

stackexchange does have some flexibility however. it would be very interesting to attempt to start a new group on area 51 that might have different rationale from the beginning, ie encourage different voting patterns to somehow officially lean toward some other criteria than the conventional one. but it takes a very large group of people to start a new stackexchange.

Answer 6

Run periodic "hunt old questions without answers" drives? Encourage such activities more? (Yes, there are a couple of interesting badges for unearthing old questions, perhaps there should be more?)