Jump to content

Wikipedia:Wikipedia Signpost/2017-02-27/From the editors

From Wikipedia, the free encyclopedia
From the editors

Results from our poll on subscription and delivery, and a new RSS feed

In January 2017, the Signpost polled its readers. We sought to learn more about our readers' habits and wishes, around subscription and notifications. We were also interested in the dynamics that bring readers to us in the first place; we believed that readers typically learn about the Signpost by finding it on their colleagues' user talk pages, but we wanted to test that hypothesis.

The poll was prompted by recent progress on a long-planned extension to Wikipedia's underlying software, which will offer a new, central page on which publications may advertise their existence, and will allow publishers to notify their readers of new issues or editions via web or email notifications instead of user talk page messages.

We also have an important (but only tangentially related) development to report. Thanks to the efforts of Evad37 and Samwilson, the Signpost once again has a functional RSS feed, here. The feed is still being refined, but is usable as of now.

Analysis

Between January 17 and February 2, 2017, we received 93 responses.

  • 54 respondents supplied usernames.
  • 61 identified themselves as subscribers; 30 said they were not, and two gave no answer to that question. (Note, we did not supply a specific definition of "subscriber.")
  • 74 listed the English Wikipedia as their "home wiki." 3 Chinese Wikipedia, 2 each from Wikidata and the French and Spanish Wikipedias, and 1 each from English Wikiquote, MediaWiki, and from the Danish, Dutch, Galician, German, Hungarian, Norwegian, and Swedish Wikipedias.

In the near term, these data will inform our decisions about the Newsletter Extension. Though it is outside the scope of our decision and our sample, the results may prove helpful to the English Wikipedia more broadly, if and when it makes a determination about whether and how to implement the extension. We were pleased with the level of response, and may run similar polls in the Signpost in the future.

How did you first learn about the Signpost?

  Another Wikipedian's user talk page, 40.2%
  On a wiki (outside user space), 40.2%
  Can't remember, 13%
  Mailing list
  Social media
  Somebody told me

Wikipedia users often learn from each another through interactions on user talk pages. When one Wikipedian sees the Signpost on a colleague's user talk page, that may be roughly analogous to noticing a magazine sitting on their table; when a Signpost notification appears in the watchlist, that may be similar to seeing a newspaper on a friend's doorstep.

The Signpost enjoys strong readership and community engagement; but that's not something we can take for granted. We therefore wanted to learn more about how our readers originally learned of the Signpost. 40% of poll respondents learned from a user talk page, which would not be part of a system based on the new extension. Another 40% learned of the Signpost on a wiki; while the extension would offer an on-wiki list of newsletters, there is no easy way to predict behavior patterns around visiting a page that doesn't yet exist. Only with extensive research (beyond the remit of either the Signpost team or the Newsletter Extension team) might we develop a strong theory about how Wikipedians might become aware of the Signpost (or other newsletters) if the delivery methods were to change substantially.

We interpret this as a strong reason to approach with great caution any changes to delivery that would eliminate user talk page notification.

How do you usually find a new edition of the Signpost?

  In my own user space, 51.1%
  On wiki (watchlist or central pages), 23.9%
  Email, 10.9%
  Somebody else's user space, 7.6%
  Social media, 6.5%

We were curious about what attracts our readers' attention when we publish a new edition. Since our main subscription options involve delivery to user talk pages and updated information in user page templates, we were not surprised to see more than half of respondents are alerted within their own user space.

One noteworthy result is that 7.6% learn that we have published a new edition via somebody else's user space, echoing the results of the previous question. Interestingly, three of the seven respondents who gave this answer also described themselves as subscribers. We would not have expected this as the primary answer from readers who identify as subscribers. This suggests that to some of our readers, the appearance of the Signpost in a familiar place may be part of the process that draws them into our pages, in addition to the formal notification that results from subscription.

What's your preferred way to receive the Signpost?

  User talkpage delivery with story titled laid out (as currently offered), 55.4%
  Single notification via "Echo", 28.9%
  Table of Contents, with links, via email, 8.4%
  Single link to new edition, via email, 7.2%

55% of respondents prefer notification on their user talk pages, as currently offered. While it's important to consider the role of selection bias, this is an especially strong result, and one we cannot afford to ignore. If even a few of our readers (say, 10%) preferred to have user talk notification, it would be difficult for us to justify doing away with it; but this goes far beyond a significant minority. Defying the preference of a majority is not a reasonable option, meaning that the Signpost cannot consider eliminating the present delivery method for the foreseeable future.

It was interesting to learn that 16% of our readers would prefer to receive the Signpost by email. This leaves an open question; since we do send notifications to two email lists (WikimediaAnnounce-L and Wikimedia-L), we don't know without further inquiry how well we are meeting the demand for email notification. If readers would prefer a direct email notification apart from those lists, that is something we may wish to consider in the future.

Our subscriber list has always been publicly visible. Should this continue?

  I don't care, 76.7%
  Yes, keep it public, 17.8%
  No, it should not be publicly visible, 5.6%

Throughout most, if not all, of the Signpost's history, we've maintained publicly visible subscriber lists. (Those wishing to subscribe privately do have alternatives, however, such as subscribing to one of the email lists noted above, adding the Signpost issue page to their watchlist, etc.)

While we've heard no complaints about subscription privacy, we did learn that keeping subscriptions private was a goal of the extension's design team, so we included this question. We also considered that, while publications have historically used subscription methods that are at least somewhat private, many modern digital publications (such as Medium, Facebook, and Twitter) treat public expressions of interest and affiliation as a feature, not a bug.

Only 5.6% of respondents preferred that the subscription lists be kept private. We hope our current menu of options (including publication to two email lists) is adequate for those readers, but can't be certain without further inquiry.

How important is it to see our story titles as links on your talkpage?

  Somewhat convenient, 30.4%
  Very important, 28.3%
  Unimportant, 25%
  I don't care, 16.3%

58.7% prefer to have the titles and links to each section visible in their notifications.

Conclusions

The primary purpose of this poll was to inform the Signpost's plans: should we anticipate transitioning to the new, Echo-based Newsletter Extension if and when it becomes available on English Wikipedia? If so, should we do so at the earliest opportunity, or wait? Should we make a clean switch, or use both the old and new methods during a transition period?

Based on our analysis of the results, we do not plan to use the Newsletter Extension in the foreseeable future. We do not see evidence that our readers have a significant problem in need of a solution (nor do we have a significant problem publishing under the current system).

We also feel that the risk of disrupting the notification patterns, as well as the risk of disrupting the dynamics that lead new Wikipedians to encounter the Signpost in the course of their normal editing process, outweigh any potential benefits. Some specific concerns:

  1. Confusion for readers, or potential readers, if they encounter a page that purports to offer a comprehensive list of newsletters, and of ways to subscribe to them, but leaves out some newsletters and methods;
  2. It would tax our limited volunteer resources to take this on, to get it right, and to maintain an additional notification method during a transition period;
  3. Our exposure to new readers could suffer, and we have yet to see a sophisticated theory for how new readers could be exposed to the Signpost under the new system.

While our poll made no effort to reach beyond readers of the Signpost, in the absence of information about broader communities (like all of English Wikipedia, or all of the Wikimedia projects), we feel this poll may be useful to the extension's development team, and may also inform wiki projects' decisions about when, whether, and how to adopt the extension.

On these broader decisions, one point stands out: the Newsletter Extension relies on listing newsletters on a single, central page. If a wiki adopts the extension (at least, as it's currently designed), any newsletters that decline to opt into the new system will not be represented on that central page. This could have the undesirable result of increasing confusion about what newsletters exist, rather than decreasing it.

Regardless of whether and how it is adopted, we applaud the effort to develop new technical tools for MediaWiki users, and appreciate the opportunity to evaluate it for our needs.

Errata

Our pie charts, and their underlying data, simplify the responses to some degree; we changed the wording of some responses to establish clearer patterns (e.g., changing "es.wiki" to "Spanish Wikipedia" so the two would be grouped together under the "home wiki" question, and combining "meta change list" with "elsewhere on Wikipedia", renaming the result to "elsewhere on wiki", for the "How did you first learn about the Signpost" question.) For transparency, each pie chart's description page on Wikimedia Commons links to both the underlying data, and to the more granular pie chart with answers exactly as provided (but with usernames redacted for privacy).

For a complete list of the original poll questions, as well as a chart of the pros and cons of various delivery methods, see below.

Original poll details

How should we deliver the Signpost? Signpost subscription poll; please submit answers by January 31, 2017

  1. Your Wikimedia username (optional)
  2. Your home wiki
    • English Wikipedia
    • Other:
  3. Are you currently a Signpost subscriber?
    • Yes
    • No
  4. How do you usually find a new edition of the Signpost?
    • My own user talk page
    • Somebody else's user talk page
    • Watchlist, not a user talk page (e.g., Signpost main page)
    • Social media
    • Other:
  5. How did you first learn about the Signpost?
    • Another Wikipedian's user talk page
    • Elsewhere on Wikipedia
    • Another Wikimedia site
    • Social media (Twitter, Facebook, LinkedIn…)
    • Other:
  6. How important is it to see our story titles as links on your talkpage? (Table of Contents)
    • Very important
    • Somewhat convenient
    • I don't care
    • Unimportant; I just click through to the main page
  7. What's your preferred way to receive the Signpost?
    • User talkpage delivery with story titles laid out (as currently offered)
    • Delivery of the Table of Contents, with links, via email
    • Delivery of a single link to the new edition, via email
    • Single notification via "Echo" (the menu next to your username at the top of the screen)
  8. Our subscriber list has always been publicly visible. Should this continue?
    • Yes, keep it public.
    • I don't care one way or the other.
    • No, it should not be publicly visible.

Considerations from MediaWiki discussion

From mw:Topic:Tit9gtsmop8qd2d8:

Method Pro
boldface = feature preserved in extension
Con
boldface = fault fixed by extension
User talk notification
(currently used on enwp etc.)
  • Immediate notification via Echo.
  • Immediate notification via watchlist.
  • Publishers can customize presentation. For instance, the Signpost, the Bugle, and the Education Newsletter summarize & link specific articles. Actualités just links each edition.
  • After initial notification, subscribers can leave the talk page notification on their page for future reference for an arbitrary length of time. (Roughly equivalent to leaving a magazine on your coffee table until it's time to discard or file it.) Different users, different preferences. Editions with personal value might be kept longer than others.
  • Many non-subscribers, including new users, are exposed to newsletters via user talk pages on their watchlist. (Roughly equivalent to finding a magazine at a friend's house or coffee shop.)
  • Subscriber list is public. (Either "pro" or "con" depending on perspective.)
  • User talk page may become cluttered if not carefully maintained.
  • Impractical to subscribe without publicly declaring subscription (watchlisting main page is a workaround)
  • Notification via MassMessage is cumbersome for publishers, and requires a user right that may be difficult to obtain.
  • Publication of a newsletter can spam the watchlists of editors who watch the userpages of many other editors. (if they keep bot edits visible) (Either "pro" or "con" depending on perspective; now listed in both columns.)
Echo web notification
(implemented in extension)
  • Immediate notification via Echo.
  • The Echo link points to a central/canonical table of contents, rather than mass-replicating the TOC. (This impacts some, like the Bugle, but not others, like the Signpost, which uses transclusion.) This makes updates/fixes easy. (Do updates ever happen? -PF)
  • Centralized-content, makes discussion easier and perhaps more likely. (Are there newsletters that publish content to user-talk, rather than just links? If so, which, and let's find out why they do it? -PF)
  • Publication is easy; does not require advanced user rights.
  • User talk pages/archives are not "cluttered up" with announcements.
  • Cross-wiki notification is possible.
  • Non-subscribers are not exposed to contents of newsletters.
  • No facility (yet?) for cross-wiki notification.
Echo email notification
(implemented in extension)
  • Subscriber may maintain their own customized archive
  • For newsletters that publish entire contents of an issue on one wiki page, this will enable easy offline reading
  • Less cluttered formatting (sidebar, top bar absent)
  • Publishers will have no insight into email-based readership (unless implementing a web beacon, which is not likely to be met with enthusiasm in the Wikimedia community)
  • Formatting may not be identical to web presentation (?) -- complex elements like sortable tables, interactive data-based graphics, image galleries, etc. may render differently
  • No direct access to talk pages and (where applicable) transcluded comment section
Blend of User Talk & Echo
  • Permits publishers to meet wishes of diverse subscriber base, where some subscribers prefer each method.
  • Publisher must maintain two separate subscription lists
  • Publisher must do more work than currently, to publish in both ways
  • Wikimedia users will see one page that claims to be "central," but which does not actually list all options; and may not list all newsletters (if some newsletters choose not to participate in new extension)
  • No easy path for subscribers wishing to change subscription method (unless publisher makes complete transition)