Wikipedia talk:Survey 2008/Archive 2

Wiki-centric survey

Why the survey is not conducted also on other sister projects? Commons is de facto my home wiki and I cannot choose it from the options offered. There is too little (if any) interest in our work like adding categories, improving image descriptions etc. Many people complain about the 500 edits limit. And how about the languages? I have edits in about 100 wikis, contribute more regularily to about dozen and daily to six. And this does not overlap with my language knowledge. For example I know some Russian and stated it in the survey but do not contribute to ru: because there is laack of common topics between cs: and ru:. In contrast I have only very basic understanding of nl: (and never would it consider a knowledge in sense of the survey question but contribute to it daily because there is a broad field of articles commons for both cs: and nl:. There should be more detailed questions to what projects, how often, to what degree and by what means one contributes... And definitely the questions should be consulted with experienced wikipedians before launching the next survey because the current questions show total lack of understanding of our daily work. --Miaow Miaow (talk) 19:32, 30 October 2008 (UTC)

The survey was intended for Wikipedia this time. We hope to conduct surveys in the sister project such as Commons in the near future. - Naoko --Shuhari (talk) 05:30, 1 November 2008 (UTC)

Sorry

I have given up answering the survey as I think it is not very well constructed. --Anna Lincoln (talk) 19:47, 30 October 2008 (UTC)

Impersonating other users

When the survey asks for your username, what's preventing you form typing anything there? There can be no cookie-based checks, since the survey site is on a different domain. So I can pick some user that irritated me, declare they make $1M a year, and that they feel terribly sorry for not donating to Wikipedia... Dandv (talk) 01:31, 31 October 2008 (UTC)

And one can complete the survey as many times as they wish, POTY on Commons had already addressed these issues[1], using open source software. Gnangarra 02:40, 31 October 2008 (UTC)

Not the best of surveys

I have completed the survery, after thinking about giving up a few times.

My answers will misrepresent me for two reasons:

1. The closest category was still too wrong for several questions.
2. Our ontologies are not in synch, and as a result I misunderstood several questions, and then could not go back later when the wording of later questions "clarified" what might have been meant earlier.

I took too long.

A good idea, but to quote Bob Lewis of InfoWorld "What you measure wrongly, you manage wrongly".

Peet Ern (talk) 03:10, 31 October 2008 (UTC)

Great survey

IMO the survey was really good with relevant questions. Apparently it was the first of its kind so it will only get better with time.

I also thought the duration was long but understandable since we are trying to get to know the editors and users.

This survey will undoubtedly create an unparalleled level of understanding about who edits and who uses Wikipedia. EconomistBR 04:07, 31 October 2008 (UTC)

Some interim comments

First of all - thanks to everyone who is giving feedback on the survey. I have read most of the comments so far, and I'm sure the UNU-Merit team is reading as well.

On the technical issues: The survey software was developed by UNU-Merit; it is custom-built software and, while it does the job, I think for next time we'll want to either use a mature open source solution (e.g. LimeSurvey) or implement our own survey tool as a MediaWiki extension. The main problem I have with the current implementation (and which has been pointed out here a few times) is the "one question per page" approach that makes proceeding through the survey slower than it needs to be, and makes it hard for the respondent to anticipate that related questions will be forthcoming, which can distort their answers. But there are many other possible usability improvements, and to reduce bias, it might also be good idea to randomize some of the multiple choice options.

On the survey itself, the questionnaire for contributors is definitely too long (we did several rounds of cutting, but it still ended up longer than it needed to be); I am very happy that we appear to nevertheless see a very high completion rate so far. I also agree that some very interesting questions were missing this time around, and I'm taking notes of some of the suggestions here.

On the topic of sister projects, this is definitely a Wikipedia survey by design, but we do want to study the other projects as well. I think it will make sense to design more specific questionnaires for them, though.

Beyond this English version, there's also lots of room for improvement in the translation process. For example, while we translated into 20 languages, we didn't facilitate localization (adapting the survey questions to the context of a specific country/region), which means that some questions (such as demographic questions on educational background) translate poorly. And, most of the translations would have benefited from further iterations of testing & feedback to reduce the number of translation errors.

That said, so far, our data indicates that we have more than 50,000 completed questionnaires in 20 languages. That's quite remarkable, and bodes well for future surveys. I'm also confident that we will be able glean high quality data on some of the key questions we've asked (there will be some distortions, but there always are, and we'll try our best to correct for them).

Thanks for all the feedback so far, and thanks to everyone who took the survey. It's far from perfect, but I feel it's an important first step for us. :-) --Eloquence* 04:23, 31 October 2008 (UTC)

As long as you guys read the feedback and act on those things within it you think are reasonable to act on, then I'd say that this will be a "success". As others have said, this is the first survey, things will (hopefully) get better from here. I like answering surveys, so if you do more, I'll answer those too. Nickjbor (talk) 10:05, 31 October 2008 (UTC)

As per Nickjbor, despite many imperfect questions 50,000+ usable responses already in the half-time, that sounds great. Thanks for your brief summary, Eloquence, I am pleased with the promise of survey regarding other projects. --Miaow Miaow (talk) 23:32, 31 October 2008 (UTC)

"What are the reasons why you read the discussion/talk page:"

This question needs a "Other reason" text box. 70.213.147.79 (talk) (really, User:JesseW/not logged in) 06:11, 31 October 2008 (UTC)

I agree. Even more to the point, it's missing the biggest reason people are actually supposed to be using talk pages as a choice! (Discussing how to improve the article.) --Delirium (talk) 08:22, 31 October 2008 (UTC)

Typo on "Depth (the information provides a deep understanding of a topic)"

The last two choices are both: "Low quality". 70.213.147.79 (talk) 06:16, 31 October 2008 (UTC)

"How negatively do you think will the following items affect your perception of a Wikipedia article’s quality?" -- some are positive!

Some of the choices given on the "How negatively do you think will the following items affect your perception of a Wikipedia article’s quality?" question are things that I would treat as positive indicators of higher quality, e.g. "History page shows multiple revisions and multiple authors" and "Discussion page contains criticism of the article"(that means someone has read it critically -- that's a Good Thing). But there were no options to register this. This should be fixed. 70.213.147.79 (talk) (really, User:JesseW/not logged in, like the previous two comments) 06:19, 31 October 2008 (UTC)

At first, I couldn't even imagine how one could perceive "History page shows multiple revisions and multiple authors" as something negative. After some thought I now assume that those who devised this question probably had something along these lines in mind: "It's harder to check whether the author is a reliable contributor if there are many contributors to an article"? This, however, is in my opinion of course completely missing how Wikipedia works. On of the chief beauties of Wikipedia lies in the fact that it doesn't matter who the author(s) are, whether an anonymous (IP) user, some pseudo or some "Prof. Dr. mult. Dr. h.c. Reginald X. Smith-Jones"... as long as the article is properly sourced and everything verifiable through the given sources. That's much more important than the question of authorship, I think. Gestumblindi (talk) 19:58, 31 October 2008 (UTC)

"Have you ever considered contributing to Wikipedia (e.g. creating a new article, editing an article or participating in a discussion, etc.)?"

This question was asked after I already answered numerous questions indicating I already had done all those things. It looks like a mistake in the software. 70.213.147.79 (talk) 06:24, 31 October 2008 (UTC)

View the questions

I wanted to see the survey questions before deciding whether I wanted to take part but I found that I needed to actually click through the survey to do this. I found this very off-putting. I do not want to start on a road when I do not know where it is leading. I feel I could be slowly led into answering questions that I really did not want to answer. One thing I find particularly irritating about surveys in general is that I might have something in particular to say, but the questions are so structured that I am unable to say it. Since I could not see all of this survey before starting, I could not tell if it fell into this category and have decided to decline. SpinningSpark 08:08, 31 October 2008 (UTC)

Can someone answer this?

Can someone answer question above about getting an eMail message, because im stuck, and don't have a clue, wheter or not wikipedia will message me. If some one could answer this that would help, and would be much apprecihated. mcjakeqcool Mcjakeqcool (talk) 13:01, 1 November 2008 (UTC)

Reply?

I'm still wating for a reply, if someone sees this question and they have an answer, please edit and give me a answer to my question asked 2 days ago. thankyou, mcjakeqcool Mcjakeqcool (talk) 11:00, 2 November 2008 (UTC)

Language

I took this survey through a link from the Swedish WP. Although it seems like it has been translated into different languages, the link from the Swedish WP was to a survey in Enligsh. The problem for me with the survey was that there were a lot of questions that I did not understand, and therefore I gave bogus answers, cause there was no box to check for "I did not understand the question". I think a survey like this should be made in simple Enligsh, with important concepts explained so that one does not need to have English as a native language to complete it. --83.183.64.115 (talk) 15:47, 31 October 2008 (UTC)

There is no Swedish translation - IMHO it may be better to remove the message from the Swedish sitenotice than to confusingly link people to an English language survey.--Eloquence* 17:07, 31 October 2008 (UTC)

According to Sweden#Language, "a majority of Swedes, especially those born after World War II, understand and speak English" (according to the German Wikipedia around 80%) - I therefore presume that the share of Swedish Wikipedia users who are able to complete the survey meaningfully is large enough to justify the message. Gestumblindi (talk) 19:43, 31 October 2008 (UTC)

What is it about?

please answer question. Simply south (talk) 23:11, 31 October 2008 (UTC)

Problems vs Symptoms

I worry that someone looking at my answers would conclude "Oh no! Spelling mistakes give a negative impression! We need to work harder at fixing spelling mistakes." Well no, spelling mistakes are a symptom that there has been little review of the article or section. I don't care so much if the editor is an A-grade speller or not. But I do care if enough people have gone over it (and good/fixed spelling is a merely a sign of this).

So when looking at the results of this survey, don't assume the solution is obvious from the problem. E.g. starting a spell-checking patrol is the wrong solution here (for me at least) as it would only fix the superficial problem of spelling, without giving a proper review to the text (nor solving the secondary problem of indicating to the reader how well reviewed the text is). —Pengo 02:21, 1 November 2008 (UTC)

New question suggestion

A question about how many sites are are a users watchlist would be useful for finding out how committed people are about monitoring pages.Septagram (talk) 03:17, 1 November 2008 (UTC)

Maybe, but I think for many editors, there may not be a strong correlation between the number of pages on that persons's watchlist and the number of pages actually being actively monitored. For instance, I have 892 pages on my watchlist, the large majority of these are not frequently changing pages. Also, how could you tell how "committed" that person is about checking the diffs when they come up on the watchlist? Dar-Ape 06:19, 1 November 2008 (UTC)

Formal training?

"Formal training" includes kindergarten and preschool? That's absolutely ridiculous. By that logic, I can mark that I have formal training in every area... I built a popsicle house, so there's engineering, we sang holiday songs, so there's music and culture, we were taught the Golden Rule, so that knocks down philosophy...

I appreciate that there's an effort here to gather some information, but the group designing this survey needs to have formal training -- you know, REAL formal training -- in collecting demographic data. It's a start, and that's good, but it needs to be done correctly. 70.122.36.93 (talk) 04:20, 1 November 2008 (UTC)

Kindergarten only counts for the number of total years, not for whether you have training. —Preceding unsigned comment added by 134.153.216.129 (talk) 19:05, 1 November 2008 (UTC)

Tailor made survey

To reduce the time to do the survey the questions each person gets to answer needs to be tailor made relative to questions already answered. Questions on translation only apply to people who are at least bilingual for example, questions about edits done do not apply to people who have already declared they are read only users. Questions to ip editors need to be different than questions to logged in editors, which will not be necessarily the same as those an admin will answer. Some questions are valid for all users but not allMjchesnel (talk) 13:27, 1 November 2008 (UTC)

Very, very poor

I'm quite disappointed in this survey. It was difficult and timeconsuming to fill out, it inadvertently tricked me into providing significantly misleading information (e.g. about the number of hours I contribute), and it then made it impossible to correct those mistakes. I fear that the results will be heavily biased towards a small, inadvertently-selected-for subset of Wikipedia users who have the time and intestinal fortitude to struggle through the survey's poor mechanics. Further impressions on my user page. —Steve Summit (talk) 15:45, 1 November 2008 (UTC)

Survey is broken in current Konqueror

After each question you get a notification that you are re-sending a POST form. You have to confirm continuing for every single question. This is annoying. Mranderson2008 (talk) 05:04, 2 November 2008 (UTC)

Five days in

Surely, there are enough constructive comments here to justify pulling the plug on this trial? We are 60 days away from 2009, which should be adequate time to redesign the quiz, when it can be relaunched under a new name. Issues have been discussed above, and many folk have generously taken considerable time to explain where the problems lie. All their names have been collected, so I suggest, before Quiz 2009 goes live, these folk should be asked to beta test the new format, so we do get statistically valid results.-ClemRutter (talk) 13:57, 2 November 2008 (UTC)

• I was just going to ask how long this is going to remain on here. I would venture to say that most people have now seen the link, and have already taken the survey if they so desire to. Eric-Wester (talk) 17:53, 2 November 2008 (UTC)

Re-start survey is not a visible option

The browser is Mozilla. Completed said 27% at the end of the last session. This seems poorly planned and shoved down my throat. How do I bypass it? SteveTaylor (talk) 19:20, 2 November 2008 (UTC)

Verifying reliability of edit count from username?

Overall, the survey was ok, though a bit long. However one thing bothered me a lot: After aksing how much do I contribute, it asked for my username, with the following note "This data will never be shared outside the survey team, but is very helpful in ensuring the reliability of the survey." Does this mean the given username will be used for validating the editing count? At least I got that impression...

I do mostly very small edits and don't care about getting any of the credit. Therefore I don't ever bother logging in, and the real number of my edits are not going to add up with what's attributed for my username. I would have liked to comment about this at the end of the survey, but alas, there was no "general comments" -box... (There should be one in each survey, really.) —Preceding unsigned comment added by Ikajaste (talk • contribs) 20:19, 2 November 2008 (UTC)

I Liked This Survey

I don't know about everyone else. Though, I was pretty satisfied with this survey. Maybe it's because I'm a student, perhaps?

At any rate, there is really only one huge error and flaw with it: "What gender are you?" The option of "Other" SHOULD NOT be there. Do you know how many people might choose it just to simply joke around? =P It would result in unreliable statistics.

If your taking the survey, you're obviously a human. And if you're a human, then you're obviously either male or female. You're probably not going to be another. If you're a hermaphrodite, you're probably going to be more one than the other. If you're an animal... you're either male or female once more.

So... it seems really unnecessary to add an "Other" option (that could potentially ruin the statistics) - unsigned

"Pangender is a term for people who feel that they cannot be labeled as male or female in gender." WAS 4.250 (talk) 11:21, 3 November 2008 (UTC)

Yes, but just how many people around the world are Pangenders? Are we really willing to risk other people skewing the statistics for just a small population of people, who can still answer whether they're male or female based on what they remember they were born with?

I'm just saying, because when I read that question, I felt a strong temptation to choose "Other" just for the fun of it. And if I was Pangender, I'd still know what sex I am, and would ultimately go with that.

This is also a question of relevance... Why is the question itself there at all? Do we expect to find out stuff like "females to more editing than males"? If yes, then are we interested in the fact that people with different body anatomy do more editing, or the fact that people who associate themselves with the culture connected with said people do more editing. Would the question "Would you associate yourself more as a male or female" then be more relevant? I wish there would have been an option "Don't want to answer", because that would have been a much better option than "Other".

I really hate these "Male or female" questions, because they are rarely actually relevant and only cause arbitrary grouping to be used when displaying the results. And every time they appear they make the cultural distinction all the more stronger. --Ikajaste (talk) 12:13, 4 November 2008 (UTC)

I believe that one of the stated goals was to figure out whether the typical Wikipedia editor is a young adult male. WhatamIdoing (talk) 23:55, 4 November 2008 (UTC)

My comments

I'd like to congratulate the survey team on the first Wikipedia users' survey. I'm sure it will provide helpful information for editors.

As previously mentioned, there are some concerns regarding the length of the survey and the language of some of the questions. I was somewhat surprised to find a question about my income, which made me question whether I should provide any personal information. One thing that should be looked at is whether the survey assumes some knowledge users might not have. For example, I seem to remember one question where you had a choice of three-letter codes for an answer. Should we assume our users know what those codes mean?

Although the survey was quite comprehensive, I thought of a couple of questions that should be included in any future Wikipedia survey. I think we should ask what version of English the user prefers -- US, UK, Canadian, etc. The survey does ask where the user comes from, but we don't know if a user from Italy prefers US or UK English, if he or she has any preference.

I would also dig a little deeper into the question of whether the user thinks Wikipedia is comprehensible. Specifically, I would have separate questions for arts, history, mathematics, applied science, pop culture and technology. My own experience is that our history and geography articles are a lot easier to understand than our articles on math, applied science and technology, but that might just be me. -- Mwalcoff (talk) 00:04, 3 November 2008 (UTC)

Maybe I'm being too picky . . .

. . . but I don't think so. After encountering two grammatical errors in as many pages (in the English version), I decided not to spend any further time on the survey. I appreciate that it is an international survey and may have been written in another language (e.g., Dutch) originally, but these are basic errors that anyone reasonably fluent in English would spot. Inexcusable. Rivertorch (talk) 05:48, 3 November 2008 (UTC)

Wikipedia:University of Würzburg survey, 2005

Visitors to this page may be interested in Wikipedia:University of Würzburg survey, 2005.

-- Wavelength (talk) 19:52, 4 November 2008 (UTC)

Wikipedia:WikiProject Wikidemia

Visitors to this page may be interested in Wikipedia:WikiProject Wikidemia.

-- Wavelength (talk) 19:58, 4 November 2008 (UTC)

:'-(

I want to see the results. _now_!!!! I can't wait. Waaa waaaaaaaaaaaa waa :'-( :'-( :'-( ReluctantPhilosopher (talk) 20:02, 4 November 2008 (UTC)

The survey Lowered my opinion of Wikipedia overall

Most of these points have already been raised. The survey seems to have been poorly translated from another language into English. The questions are confusing and poorly written. One question per page is in no way shape or form user friendly. The answers to the questions are repetitive, highly biased, or both.

Improvements: Have multiple English speaking editors look over the survey before you post it for the world to see and judge. (Perhaps some handy Wikipedians?) Put more than one question on each page and separate it into sections. The sections could be Basic info about the survey taker, impressions of Wikipedia, Contribution (time/money) levels, etc. Remove the reputation question! Reputation has never been my motivation and, from a lot of the responses here, is not most people's motivation. At least allow the survey taker to say as much. As far as the level of use question (for Wikiquotes etc.) goes, I think it would be more helpful to add more options. I have used Wikiqoutes once, but the option I picked made it sound like I used it all the time.

If our donations go to projects like this, I sincerely hope that the next one will be less awkwardly done. (Hell, I've seen better quizzes on OkCupid, and that is saying something.)Self-made prisoner (talk) 21:53, 15 November 2008 (UTC)