Template talk:Find sources/Archive 2

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 1 Archive 2

Nomination for deletion of Template:Module:Find sources/templates/Find sources

Module:Find sources/templates/Find sources has been nominated for deletion. You are invited to comment on the discussion at the template's entry on the Templates for discussion page. {{3x|p}}ery (talk) 22:00, 15 February 2019 (UTC)

Add Library Genesis?

I increasingly find Library Genesis to be a very helpful search engine for academic papers, more so than Google Books. In particular, it excells at finding book reviews. Ex. compare [http:/blacklisted_domain_removed/scimag/?q=Groundswell.+Winning+in+a+World+Transformed+by+Social+Technologies LG] return hits and Google Scholar. I first noticed it few weeks ago while looking for reviews for this book I wrote an entry on. If anyone is concerned about legality, note that LG has been mentioned at Wikipedia:WikiProject Resource Exchange/Resource Request (with a caution note) for quite a while now, with no objections/controversy. We could add a link to such note to the template through a tiny hperlink to address any related concerns. --Piotr Konieczny aka Prokonsul Piotrus| reply here 04:30, 3 June 2019 (UTC)

Have

been planning on adding Sage and Cambridge Core. Any objections? ∯WBGconverse 06:52, 7 June 2019 (UTC)

Fine with me. No objections, just like I presume no objections to my proposal above. --Piotr Konieczny aka Prokonsul Piotrus| reply here 05:34, 11 June 2019 (UTC)

Inclusion of NYT

I don't particularly have an issue with it, but I do want to just bring up for discussion the fact that the NYT is the only newspaper included with this template. While it is probably true that the NYT has the best claim to being the newspaper of record for the English speaking world, I'm saying that as an American, and I can imagine those in other English speaking countries objecting that the NYT is not the best source for them, and that perhaps another paper should be included, or that no specific paper should be included, so that Google News (a more dynamic way to find a quality source, but also not as curated) can be used instead. I guess my point is mainly that this template in some way implies that we're endorsing the NYT over every other newspaper, and while in a practical sense that's probably the reasonable thing to do for the situations this template is used for, we should at least have some discussion to establish a consensus. (Also, as a minor cosmetic note, NYT ought to be italicized, and I'd appreciate someone with template editing privileges making that fix.) - Sdkb (talk) 05:27, 28 June 2019 (UTC)

Excluding non-Wikipedia source

I noticed the template's Google news search excludes this article, though the word 'wikipedia' does not seem to appear anywhere in it.

See: Find sources: Google (books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL

Oornery (talk)

@Oornery: My best guess as to why this happens is that when Google crawled that page, it contained the text "Wikipedia", but it doesn't contain it any more. (Maybe it appeared in the "latest articles" feed or something like that.) However, if you notice a pattern of this occurring, it might be a good idea to report it to Google, as it might be a problem with their algorithm. — Mr. Stradivarius ♪ talk ♪ 01:29, 12 January 2020 (UTC)

Internet Archive?

I use Internet Archive all the time to look for sources, especially on old topics. Would it make sense to include a full-text search of archive.org on this template? AleatoryPonderings (talk) 20:57, 23 July 2020 (UTC)

Hmm, interesting thought. I do get a lot of sources value out of the Internet Archive, but I also see it as sort of a more advanced tool less essential than some of the others listed. {{u|Sdkb}} talk 01:44, 24 July 2020 (UTC)
Fair enough—certainly a bit clunkier to use, too. Just a thought. AleatoryPonderings (talk) 02:53, 24 July 2020 (UTC)

Template-protected edit request on 24 July 2020

Please italicize NYT per MOS:ITALICS, since it's an abbreviation of the name of a newspaper. {{u|Sdkb}} talk 01:47, 24 July 2020 (UTC)

Sorry, I don't see the guidance you are referring to. Can you please quote the relevant MOS guidance? Thanks. – Jonesey95 (talk) 03:17, 24 July 2020 (UTC)
Jonesey95, sure:

Italics should be used for the following types of names and titles, or abbreviations thereof:

  • Major works of art and artifice, such as albums, books, video games, films, musicals, operas, symphonies, paintings, sculptures, newspapers,...
{{u|Sdkb}} talk 05:15, 24 July 2020 (UTC)
To editors Sdkb and Jonesey95:  done, and also in {{Find video game sources}}. P.I. Ellsworth  ed. put'r there 08:20, 24 July 2020 (UTC)

Wikipedia library

Moved to Module_talk:Find sources#Wikipedia library.

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Just a quick note that the Wikipedia Library link should point to https://wikipedialibrary.wmflabs.org/ rather than /partners, as the correct entry point for accessing content :) Samwalton9 (WMF) (talk) 11:02, 29 October 2021 (UTC)

I've updated that (in Module:Find sources/links/wikipedia library), since it's really independent of the rest of the rollout. * Pppery * it has begun... 17:27, 29 October 2021 (UTC)
(edit conflict) Samwalton9, I took another look. When I go to the domain home I'm unable to access any resources, nor even see which ones are available. A better choice, it seems to me, is https://wikipedialibrary.wmflabs.org/users/my_library/ which for a logged-in user provides a search box that when filled and submitted will execute a search on the sources they have access to. (A non-logged in user gets a WM login page, which then proceeds directly to the search page.) Ideally, we'd prefer to skip the page with the search box and just execute the search directly upon clicking the link (as we do for every other link in the find-sources set except for that one) but in the case of WP library the search is executed by ebscohost.com which requires url params that encode session data from the logged-in user and other information (unlike all the others) and so we're unable to do that. But it seems to me that going to the 'my_library' gets the user as close as possible to the WP Library results they are seeking; it still requires them to fill the search box and hit Submit, but nevertheless it's better than the domain home page. Unless there's something I'm missing, in my opinion we should use the my_library address. It's second-best compared to showing the search-results directly, but the best we can offer now. (Domain eds.s.ebscohost.com stores two session cookies, and maybe down the road an enterprising js- and template editor could look into whether session or other information could be extracted to provide the missing ebsco query params and execute the search directly, but that's well beyond the scope of this implementation.) Mathglot (talk) 18:21, 29 October 2021 (UTC)
Pppery, thanks for that. My response above had an edit conflict. Can you take another look at the module url for WP Lib in light of this information, and if Sam agrees with the reasoning, adjust the link in the module again as needed? (P.S., in refactoring this to a subsection, I dropped an indent level, thus technically a TPO-vio; hope you're okay with this.) Mathglot (talk) 18:25, 29 October 2021 (UTC)
The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Mathglot (talk) 20:50, 29 October 2021 (UTC)

Summary: 'Find sources' 2021 update

The following is a brief summary of current consensus. Please feel free to edit this list if you disagree with anything or if something changes based on discussions happening elsewhere.

Implementation: improve customization of Template:Find sources

The proposal at VPP to improve customization of this template was approved. This discussion is to track implementation, and to discuss any issues that may arise. Adding Wikmoz, and Sdkb; feel free to notify other interested users. Mathglot (talk) 00:52, 17 October 2021 (UTC)

Goals

For starters, I thought we could talk about what we are implementing. Here are the four possible "find sources" templates (FSTs) so far:

Of these, I think it was clear we are implementing the wrapper which will choose between two, possibly three of them: B (bio) and G (gen; aka, the default template, that is: {{find general sources}}[post implementation] = {{find sources}}[currently]). Maybe also V, as that template existed prior to the VPP proposal, and has a clear WikiProject associated with it. Seemed like the sense was to maybe put off B (bio) to a later stage. Does that concur with everybody's impression? Mathglot (talk) 01:22, 17 October 2021 (UTC)

I think we should implement the medical sources and video game sources initially, and leave the biographical sources for a subsequent round. That's because the former two are both more limited in scope (so better for initial testing) and have significantly more history/usage establishing them. {{u|Sdkb}} talk 02:07, 17 October 2021 (UTC)
I agree that it would make sense to initially focus on gen, med, and vg templates. It looks like the vg template is already in use in some places and the med template provides a significant improvement over the current source list. The bio source list seems to have the most open questions regarding sources to include. - Wikmoz (talk) 04:00, 17 October 2021 (UTC)
Forgot one; added 'mil' to the list above. Like, 'biog', This falls into the "not right now, revisit later" bucket. Mathglot (talk) 18:08, 17 October 2021 (UTC)

The other thing I think we decided was to go with an automatic, WikiProject-detection scheme, overridable by opt-in usage of a param, such as |search-domain= or similar. - Mathglot - 01:36, 17 October 2021 (UTC)

Mathglot, Sounds good. The override parameter would just be added to the {Talk header} template, correct?
Talk:Medtronic: {{Talk header|search_term=Medtronic|search_domain=general|archive_age=365|archive_bot=lowercase sigmabot III}}
Seems easy enough for editors to manage. - Wikmoz (talk) 19:44, 19 October 2021 (UTC)

An issue was raised about auto-detection in some cross-project topics, such as medical device companies, for which med-only links would be too restrictive, and we'd either want the G-links, or both sets. Wasn't clear to me whether we decided to deal with that issue in the first implementation, or only with refinement after the first one. Mathglot (talk) 01:46, 17 October 2021 (UTC)

I'll add that to the side issues list. I don't think there were any objections to deferring that to a future update. - Wikmoz (talk) 04:00, 17 October 2021 (UTC)

Where the talk pages reside

Note: this page was previously a redirect to Module talk:Find sources. With the implementation of this proposal, and especially insofar as the wrapper template is concerned, the topic and interests of this talk page and the Module talk page diverge sufficiently that they have been uncoupled again. Once this implementation is complete, and any follow-up issues are taken care of, a discussion can be had at some future date as to whether it makes sense to merge the talk pages again. Mathglot (talk) 00:55, 17 October 2021 (UTC)

As the editor who performed the original talk page merge, I agree that it makes sense for the wrapper template to have a separate talk page. That said, Template talk:Find general sources, Template talk:find biographical sources, Template talk:find medical sources and Template talk:Find video game sources should IMO be merged to Module talk:Find sources, as all four templates are wholly dependent on the module. * Pppery * it has begun... 03:00, 17 October 2021 (UTC)
Makes sense to me. I think *during* implementation and wrapper installation the four TPs should remain here in case there are individual issues with them vis-a-vis the wrapper; it would be awkward to jump back and forth to deal with that. But there (hopefully) won't be any issues, and in any case, it would be a temporary situation, and once implementation is complete (or even before that: once it's clear there are no implementation issues that affect them) they could/should be merged to the Module TP. Mathglot (talk) 20:53, 17 October 2021 (UTC) updated to add a new subsection hdr; by Mathglot (talk) 20:56, 17 October 2021 (UTC)
I've now redirected all of them except Template talk:Find medical sources (since I've seen a lot of arguing specific to medicine, which is taking place elsewhere but could easily go on that talk page) to the module talk. * Pppery * it has begun... 17:43, 16 November 2021 (UTC)

Wrapper issues

The wrapper is here: {{Find sources/sandbox}}. It uses helper subtemplate {{Find sources/top proj}} which probably needs a rename. There's a mocked up usage of the sandbox wrapper here: Talk:Cochlear Limited/Talk header test. Mathglot (talk) 01:34, 17 October 2021 (UTC) updated by Mathglot (talk) 01:40, 17 October 2021 (UTC)

Explanation of some wrapper design considerations: the core of WikiProject autodetection was moved out of the wrapper sandbox to subtemplate {{Find sources/top proj}} some time ago. Neither of these two templates is very long or that complex as far as nesting or squirrely code is concerned, but the concepts of what is going on functionally are a bit complex, and for the sake of future template editors I thought it best to divide the functionality this way. This localizes all of the project autodetection in subtemplate "top proj", while the wrapper concentrates on picking between project autodetect and search-domain override (and possibly other methods in the future).

Another design consideration targeting clarity and future maintainability is the liberal use of newlines and indentation in the wrapper and subtemplate wikicode for alignment of conditionals and switches, taking care that no unwanted white space is introduced into the output. This should also ease understanding of what's going on, and facilitate upgrades and maintenance going forward. I would hope that other editors would continue in the same vein, but that's not up to me, of course.

The name "top_proj" was chosen to suggest the fact that the "top priority" project is returned (not to be confused with the one closest to the top of the Talk page). This means that "top_proj" implements a pecking order among WikiProjects, based on which one is the "most important" and should trump the others if two or more of them appear on the same page. This order is currently: 1. Medicine, 2. Biography, 3. Video games. Note that top_proj *does* return the biography token (if the article doesn't also belong to Proj Medicine) but the wrapper does not currently recognize 'biography' and will generate "general" source links in this case. This is by design and another benefit of the decoupling of project detection and wrapper domain choice, as the two do not need to be "in sync"; In this case, top_proj is "ahead" of the wrapper, and so if we decide to add biographical source links in a subsequent upgrade, only the wrapper will need to change. (That said, 'biography' should probably be swapped with 'video games', because currently an article which belongs to both 'Biography' and 'Video games' projects—such as Brianna Wu—will return token 'biography', which the wrapper will ignore, and use the default 'general' links instead; swapping them would return video game links instead.) It's not impossible that other methods of choosing among WikiProjects could be implemented in the future ("highest_proj", or whatever), and in particular, stacked link sets based on multiple projects has been discussed as a future upgrade possibility, as is noted in the #Status summary below.

Probably some of this information should be copied someplace (doc pages?) for the benefit of future template editors, but I wanted to record it now while it was still fresh in my mind. @Wikmoz and Sdkb: (I try to use pings judiciously, but if you wish to stop being pinged to this page because you're already watching it, or maybe only when I update something other than the last section, lmk and I'll try to observe your preferences.) Thanks, Mathglot (talk) 19:34, 27 October 2021 (UTC)

Thanks for the clarification! I'll update the summary section accordingly. I'm watching this page but in general, no objections at all to the extra pings. - Wikmoz (talk) 20:36, 27 October 2021 (UTC)

Positional search_terms bug: Wikmoz identified a problem in testing, where use of search_terms beyond the first one are not handled by template {{find medical sources}} (it wasn't designed to; ditto 'general' and 'biographical', in the same way). The reason for the problem is identified, and two approaches are possible. Details at #Final test and rollout planning below. Mathglot (talk) 18:38, 29 October 2021 (UTC)

Side issues

Some other issues were mentioned or discussed, either at the VPP or separately. We could list these here, and if it affects this implementation, discuss as needed.

Newspaper of record

  • the "newspaper of record" thing. Mockup (?) at Talk:Koromiko, New Zealand. Mathglot (talk) 02:06, 17 October 2021 (UTC)
    This is pretty separate from the rest of this, so I'm planning to propose it once we're done with the implementation here. The side issue I'd like to see resolved sooner is removing Google Newspapers (≠ Google News), as it'll get the find sources module back on one line (it went to two on my New Vector display after the change from WPL to "WP Library"). I put an invite to get more participation on that, and if there are no further replies, I'll implement in a day or two. {{u|Sdkb}} talk 02:12, 17 October 2021 (UTC)
    Seems like a nice but low priority feature that can probably be deferred. - Wikmoz (talk) 04:06, 17 October 2021 (UTC)

Link sets from multiple domains

I wrote up a subtemplate that could be used as part of the implementation of stacked lists going forward. {{Find sources/proj list}} examines the talk page for a given page (default: the page it is on), and returns a CSV string with a set of tokens indicating which projects it found. If more than one project is found, it returns a token for each one. (It currently isn't really a csv, because it includes a trailing comma, but that could be trimmed if desired.)

  • Example: {{Find sources/proj list|Marie Curie}} = medicine,biography,military,

To implement stacked lists, the wrapper would get the tokens, and then generate the {{find * sources}} up to however many are desired, substituting each token for the asterisk. Mathglot (talk) 21:28, 29 October 2021 (UTC)

Impact on other templates

  • Impact on other template usage – implementation may affect the usage of other templates. I noticed the presence of Template:Reliable sources for medical articles at Talk:Gender dysphoria for example, and it's doubtful if it would still be needed post-implementation. It currently has 13,910 transclusions. Mathglot (talk) 05:30, 19 October 2021 (UTC)
    In the short term, I don't see any problem with them coexisting. If the template is eventually revised to focus more on MEDRS guidelines and less on resource suggestions, it really would complement and not compete with Find sources. If eventually merged, I guess it would be easy enough to add a "preamble" text field to the Find sources template. The only issue I can think of is that in the initial proposal, someone raised the issue that MEDRS might only apply to a subset of medical topics (e.g. it wouldn't apply to a medical company). In this case, it may be better to keep the MEDRS instructions in a separate template. Either way, feels like something that can be addressed longer term. - Wikmoz (talk) 20:47, 19 October 2021 (UTC)

General sources notice

{{General sources notice}} is part of a suite of such templates which includes {{medical sources notice}}, {{biographical sources notice}}, and {{Military sources notice}}. Strictly speaking, these are outside the scope of the wrapper template and the {{Talk header}} template, but are a closely related adjunct to them. I see a few use cases for them:

  1. on article talk pages that don't have talk header templates – A large number of article talk pages do not have a talk header at all (to me, a surprisingly large number). Although it's nice that Template:Talk header incorporates "find sources" behavior now, that leaves many articles that do not benefit from automatic placement of "find sources" because there is no Talk header. (I've occasionally run across some pushback when adding Talk headers to pages that didn't have one.) This is perhaps the main use case.
  2. on Talk pages where you might want two or more, and you wanted to control the domain sequence and which ones
  3. on Talk pages where you might want to decouple "find sources" from the header – by using the talk header param |hide_find_sources=yes to turn off automatic links in the header, you could choose which find sources domain(s) you wanted, and place them lower (or higher) in the sequence of banners appearing on the talk page.
  4. in article discussions, where a user might want an alternative to the {{find * sources}} series which can be used inline without highlighting or a new paragraph, whereas the boxed version sets it off visually and in block mode.

Currently, the "notice" templates include a small icon at the left, and I was planning on tailoring them to the domain, so that medical notice gets the rod of Asclepius, video would get a little video cam icon, and so on, as an additional visual signal to finding the right set. However, the "find sources" links that are generated by the {{talk header}} template do not have such icons, so there is a mismatch in how they are handling it currently. I'd like to see the icon retained, including added to the Talk header template (possibly as a background image left, so it doesn't take up any horizontal space) but I suspect consensus would be against that. Mathglot (talk) 21:26, 27 October 2021 (UTC)

I understand now. And yes, it will be great to have domain icons pass through to the Talk header template. Would there be any organizational gain from having a single 'Sources notice' template that leverages the domain parameter (e.g. {{Sources notice|domain=med}})? Or I guess that gets a little unwieldy for editors to use in practice. - Wikmoz (talk) 02:50, 29 October 2021 (UTC)
Hm, interesting suggestion. I don't think it necessarily has to be either–or; we could keep the original notices templates, and also have a wrapper {{Sources notice|domain=med}} that works the way you suggest. Mathglot (talk) 02:02, 30 October 2021 (UTC)

Module integration

I'm frankly not sure what must, or should be ported or changed in the module, once the template-based changes are live and working to our satisfaction. I don't think we have to decide that now, but if there are issues that ought to be thought about while doing the template-based implementation, we can discuss it here. Mathglot (talk) 01:36, 17 October 2021 (UTC)

This seems to be happening bit by bit while resolving some of the bugs or other issues anyway. So much the better, probably; there will be less left over to do later, if anything. Mathglot (talk) 02:56, 30 October 2021 (UTC)

Testing issues

I'm seeing multiple testing issues which may be artefacts of the testing environment and interfere with normal testing. I'm going to try to identify them here, and see which ones are real and which aren't, show how to reproduce them, see if there's a workaround, and analyze whether it's a real problem or not.

These issues may show up differently depending on the test environment used:

  1. Special:ExpandTemplates
  2. Template:Find_sources/testcases
  3. Template:Talk header/testcases
  4. in situ Talk page tests,[a] as described at #Group 4 – WikiProject autodetection and #Group 5 – WikiProject override in Template:Find_sources/testcases, and in sections #WikiProject detection and #WikiProject override in Template:Talk header/testcases.

In brief, I'm seeing tests that fail in Special:ExpandTemplates, may work in Template:Find_sources/testcases, but fail in the test cases in Template:Talk header/testcases but are okay in in situ tests.[a] Mathglot (talk) 22:53, 29 October 2021 (UTC)

Script timeout on archive list

The first thing I noticed, is that some environments like #1 (ExpandTemplates) got into a strange loop trying to expand {{Talk header}} and displaying dozens or hundreds of archive links that didn't exist, and finally ran into a script timer limit, and stopped expanding the template, so that the "find sources" links would never get shown at all.

  • Reproduce: Invoke {{Talk header/sandbox|demospace=1|search_term1=Marie Curie}} in Special:ExpandTemplates
  • Observe: Displays about 100 red wikilinks for archives 'A' through 'GZ', the dies with error 'The time allocated for running scripts has expired.'
  • Note: this is not due to sandbox code; {{Talk header|demospace=1|search_term1=Marie Curie}} dies in the same way.
  • Workaround: add |noarchive=yes: Invoke {{Talk header/sandbox|demospace=1|search_term1=Marie Curie}}
  • Result: the talk header is displayed; so the workaround works; (but sandbox version doesn't show the correct search term, whereas live[a]test does.)

Different behavior in ExpandTemplates and in situ

The test above that fails in Special:ExpandTemplates passes in an in situ test at Talk:Marie Curie.

  • Reproduce: run {{Talk header/sandbox|demospace=1|search_term1=Marie Curie}} in a) Special:ExpandTemplates, and b) in Talk:Marie Curie (by editing it and using Preview mode)
  • Observe: a) fails: (how?[b]) but b) passes.
  • Note: a) succeeds, if you use the live and not the sandbox version, but that takes a completely different path.
  • Workaround: it may be that we have to avoid using Special:ExpandTemplates to test the sandbox version of {{Talk header}} because of misleading results; but I don't yet know why it acts differently.
  • Result: in situ testing seems to be more reliable; here's another one (Note: in situ tests do not require |demospace=):
when tested in situ this displays medical source links, with all three search_terms in place, and matches these links from the standalone template invocation {{find medical sources|Marie Curie|radium|Nobel}}:
Find medical sources: Source guidelines · PubMed · Cochrane · DOAJ · Gale · OpenMD · ScienceDirect · Springer · Trip · Wiley · TWL

Conclusion: the ExpandTemplate tool doesn't reliably display the /sandbox version of {{Talk header}} for some reason, and we should do in situ tests for this instead. (A summary of this should be added to Template:Talk header/testcases as well.)

more coming...

Notes

  1. ^ a b c live and in situ: I'm using the word live to mean what happens now on an article Talk page, without any mucking about with the content in Preview mode; I'm using in situ test to mean taking a published Talk page and altering it with some test like adding a sandbox template instead of the current one, and see how that behaves.
  2. ^ a) fails: (how?): mouse over Google books (or another url) in the sandbox invocation and notice that the search keywords are: "ExpandTemplates" -wikipedia, whereas the live version it searches "Marie Curie" -wikipedia.
I've been doing my testing on Preview pages and as everything looks good there. FWIW, I tried {{Talk header|demospace=1|search_term1=Marie Curie}} (no sandbox) in Special:ExpandTemplates and got the same endless archive list timeout error. I agree that the ExpandTemplate quirks aren't a blocker so long as things look good in situ. - Wikmoz (talk) 04:07, 30 October 2021 (UTC)

Final test and rollout planning

@Sdkb and Wikmoz: I've added changes to the wrapper in {{find sources/sandbox}} to include project autodetect override via the |domain= param (alias: |search-domain=), and modified {{Talk header/sandbox}} to use the find_sources/sandbox (for now; will need updating at cutover to the main template) and to pass the domain param through. Also, the test cases at Template:Talk header/testcases have been updated, and Template:Find sources/testcases has had a major expansion. (Both still new and unchecked so far, so may be buggy or with unclear documentation.)

Am thinking a few days to check it out via discreet testing, then cut both over at the same time, and notify WP:MED of the release, so they can watch for any bugs or anomalies. For right now, you can test in Preview mode, or alter the Talk header in a few live articles that are isolated or not viewed very much; just change the current Talk header invocation to the sandbox version ({{Talk header/sandbox}}) with or without the |domain= param.

Example: to allow Talk:Giardiasis to do WikiProject detection, change the first line to:

  • {{Talk header/sandbox}}

Expected result: medical source links are shown in the header.

To force Talk:Giardiasis to ignore override WikiProjects, change the first line to:

  • {{Talk header/sandbox|domain=general}}  – (or even, |domain=video)

Expected result: ignores the project, shows general links (or video links), as specified by the param. For more details, see the test cases pages.

Thanks, Mathglot (talk) 07:29, 27 October 2021 (UTC)

Thank you, Mathglot! Will test today. - Wikmoz (talk) 17:34, 27 October 2021 (UTC)
@Wikmoz:, please excuse any muddiness in the test doc; I was pretty tired completing it, and I expect there will be areas that are unclear, incomprehensible, incomplete, or even wrong. If you can figure out what it should've said (or you feel like adding more test cases) please go right ahead. If it's completely opaque, just let me know (at the testcases talk, preferably, or here) and I'll fix it up. Thanks! Mathglot (talk)
Really nice status summary, by the way; I came here again to say something about stacked lists not being included in the first rollout, but I see you've already covered that there. It's handy to have everything in one place like that, so thanks again. I've added a header over this subsection to set it off. Mathglot (talk) 18:32, 27 October 2021 (UTC)
So far, so good on testing. It looks like the template also gracefully handles illegal entries, which is nice. Random question... Was Template:General sources notice just for testing? The equivalent Template:Find sources notice was deleted. - Wikmoz (talk) 20:48, 27 October 2021 (UTC)
(edit conflict) Good question; that deserved some additional explanation, which I've added under the #Side issues subsection; see new subtopic #General sources notice. Mathglot (talk) 21:29, 27 October 2021 (UTC)
Regarding variables, and feel free to disregard as this is probably in nitpicking territory, is there a benefit to introducing single letter variables? "med" and "medical" seem like they'd be sufficient without the single letter option. It keeps the variable meanings self-explanatory to anyone who encounters them and avoids future conflicts (e.g. "m" / "mil" / "military"). For video games, "vg" may also be preferable to "v" and "video". - Wikmoz (talk) 21:17, 27 October 2021 (UTC)
That's a good point. Single letters is just my accommodating people who are laconic or lazy (where's that mirror?) but your points trump that. Let's add this to the list of things to be changed. (If you're comfortable changing the template sandbox and/or the testcases, feel free.) Mathglot (talk) 21:31, 27 October 2021 (UTC)
Sounds good. I updated this on the test case pages and sandbox. I didn't adjust {{find sources/top proj}}. - Wikmoz (talk) 22:28, 27 October 2021 (UTC)
Ack. I changed "video" to "vg" in the template sandbox. Your edit to the test cases page restored "video". Should I restore it to the template? - Wikmoz (talk) 22:40, 27 October 2021 (UTC)
 Fixed – No worries; did it already. We can talk about it, but I like 'video' in there, because it's the only one-word choice for that one. And when I think about the sources, I think in my head, "find video sources" even though what I really mean is, "find video game sources", and if I think that way, probably I'm not the only one. Mathglot (talk) 22:46, 27 October 2021 (UTC)
Thanks! - Wikmoz (talk) 22:53, 27 October 2021 (UTC)
Bug: The 'search-domain' alias appears to work on the test page but when I try it on an actual talk page, it generates an error: "Preview warning: Page using Template:Talk header with unknown parameter "search-domain"". Swapping the parameter name to 'domain' eliminates the error. - Wikmoz (talk) 22:14, 27 October 2021 (UTC)
 Fixed – thanks; that was on my list of things, and I got distracted responding here, lol... Mathglot (talk) 22:24, 27 October 2021 (UTC)
Bug: The parameter 'search_term' is ignored in the sandbox. The issue is visible in the test cases on Template:Talk header/testcases. - Wikmoz (talk) 00:23, 28 October 2021 (UTC)
 In progress – Good catch! Hm, nothing obvious from reading the code; will have to regenerate it and see what's happening. Mathglot (talk) 00:49, 28 October 2021 (UTC)
Latest: there's a problem in the way the wrapper passes arguments through to the four domain-specific templates, and the way they handle them. Mathglot (talk) 05:39, 28 October 2021 (UTC)
Ugh. I hope it's not too much of a headache to figure out. - Wikmoz (talk) 18:38, 28 October 2021 (UTC)
Are there any other template experts we can ping who might be able to help debug why the arguments are being dropped? Wikmoz (talk) 21:01, 28 October 2021 (UTC)
@Wikmoz: It's clear why the args are dropped; the find * templates don't handle positional args 2-4 like the Module does, only arg1; (they weren't intended to) and just need to have them added. On the other hand, it's mysterious to me why the Module allows five (or more) positional args for the search_terms, since they all get joined by white space anyway, and params 2 through 5 could just as well have been one param with multiple search terms, i.e, |2=term2 and |3=term3 and |4=term4 and |5=term5 is equivalent to |2=term2 term3 term4 term5 when doing a search for them.
There are two ways to solve that, and the first is just to expand the templates (notably find medical sources) to specifically deal with args 2 through 4 . The other is to switch gears entirely, and convert it to use the Module subtemplates, a lot of which it looks like you have already defined. That is a bigger change functionally, but is closer to the ultimate destination of all this. So, I'm kind of mulling over which way to go, and in the meantime, I just need a break from it for a day or two to do other stuff.
So there's no blockage here, just deciding which way to go with it. If you want to help, you can: just see if the Module has all the config and doc subtemplates that it needs for {{find medical sources}}; video was converted long ago and is already handled. I'll be back on it soon. Mathglot (talk) 07:13, 29 October 2021 (UTC)
Setting up the modules for Template:Find medical sources... I'm just unsure how to fix the Examples section in the documentation. - Wikmoz (talk) 18:42, 29 October 2021 (UTC)
I noticed your change, nice job. I don't know either about the doc. A first glance shows that that fixes test case group 3-2* but I haven't checked beyond that. Mathglot (talk) 19:00, 29 October 2021 (UTC)
It looks like the autodoc calls for Template:Find sources medical instead of Template:Find medical sources. I set up a redirect to fix (mirroring what was done for video games) but this seems like a bad solution. I think the fix needs to be made in the autodoc. Also, it looks like we're stuck with the "Albert Einstein" and "Happy Birthday (Stevie Wonder song)" example searches. - Wikmoz (talk) 19:31, 29 October 2021 (UTC)
We may be stuck with those examples for now, but I don't think we should just accept that situation permanently. In the end, there has to be a way of including the examples we want; if need be, the autodoc can be adjusted to slurp a template with individual examples for each one, and that can be another flavor of submodule config. But it's not the most pressing thing right now, and we should add it to the side issues to be looked at afterward. Mathglot (talk) 20:14, 29 October 2021 (UTC)
This resolved the search_term bug for the medical template but I still see it in the general template. Just an artifact of the current setup? - Wikmoz (talk) 20:47, 29 October 2021 (UTC)
There are some confusing testing issues going on, that I'll address separately as it may affect how we need to test. I just started #Testing issues (above this) and will expand it as I discover issues. Mathglot (talk) 22:53, 29 October 2021 (UTC)
I think so; I noticed that it resolved for medical (and I think also for video, but I haven't checked yet) but not for the general template. I have to see if that's due to the fact that the invocation config is different than the template name itself; this may be related to what I mentioned at #Wrapper cutover issue. Mathglot (talk) 01:22, 30 October 2021 (UTC)
I changed something; try now. Mathglot (talk) 01:25, 30 October 2021 (UTC)
Success! Looks good in my testing! One related question: in Template:Find sources/testcases, under Group 2, there are test cases that use the search_term= parameter... but I thought this parameter is for Talk header. - Wikmoz (talk) 03:47, 30 October 2021 (UTC)
You're right; that sounds like an oversight on my part; as you point out, there's no such param; I'll have to go fix those cases. Mathglot (talk) 17:44, 30 October 2021 (UTC)
I resolved this on the test case page. - Wikmoz (talk) 18:36, 30 October 2021 (UTC)

Bug: An article which belongs to both 'biography' and 'video' projects displays only 'general' sources.

 Fixed This case was identified in #Wrapper issues (here). Fixed by changing {{Find sources/top proj}} in this edit to ensure 'video' is returned for those articles in both projects,
 Checking... This is a change to a helper template called by the wrapper, which means that all test cases need to be briefly reviewed, to make sure the fix didn't break something.

Also, probably need to add some 'pecking order' test cases; i.e., if something is in both 'med' and 'video' which set of links is shown? (This whole issue may go away when stacked links is implemented, or maybe it morphs into which set is displayed on top.) Mathglot (talk) 18:17, 30 October 2021 (UTC)

Bug?: It looks like there's some logic somewhere to remove "Talk:" or anything preceding a colon from the page name when generating the search term. You can see the difference in Template:Find sources/testcases. Maybe there are wildcards in production that are not in the sandbox? Visible here as well:

Find sources: Find sources: Google (books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL
Find sources/sandbox: Find sources: Google (books · news · scholar · free images · WP refs) · FENS · JSTOR · TWL

Not a significant issue but at a minimum, worth tracking if it's not easy to fix. - Wikmoz (talk) 18:36, 30 October 2021 (UTC)

 In progress Another good catch; that is significant because it could alter the searches, and should be fixed. I'll take a look at that. Mathglot (talk) 02:05, 31 October 2021 (UTC)
 Fixed Actually, not important for cutover of this functionality on article Talk pages because the previous version already stripped Talk prefixes, but it did affect searches in others spaces like Template, Template talk (like this page), Wikipedia talk, and so on. An easy fix, now complete. (Notice the links above are equal, now.) Nevertheless, since this represents a change to the wrapper template, we need another round of verification. Mathglot (talk) 04:19, 31 October 2021 (UTC)
Thanks! Will retest. - Wikmoz (talk) 05:16, 31 October 2021 (UTC)

Bug: well, more of a config issue than a bug, really: the links generated by the live template and the sandbox are slightly different: one has "newspapers" link; you can see this in the example you added above, where the sandbox version includes "newspapers" but the live one does not. Since template {{find general sources}} (which provides the bottom set of links) was an exact copy of {{find sources}} at the time the copy was taken (or more exactly, when the module config was set up for it), this means that {{find sources}} (or rather, it's module config) must have moved on since then. Checking the history of Module:Find sources/templates/Find sources we can see that indeed it was changed on October 17, when the google newspapers link was removed (diff). That explains why all the other test cases that have nothing to do with the wrapper changes have been failing, like all the Auto archiving notice tests. Updating the Module config for general sources to match the other one by removing the "google newspapers" link should make all the failing tests pass again. Mathglot (talk) 04:55, 31 October 2021 (UTC)

 Fixed There was one other difference as well, but they're all fixed; all the yellow archiving tests are back to green again; the only yellow ones remaining are for the new functionality which is as expected. I'm done for today but will check all the tests and links tomorrow. Mathglot (talk) 06:24, 31 October 2021 (UTC)

Everything looks good on my end. Any other issues still in progress? - Wikmoz (talk) 18:19, 31 October 2021 (UTC)

I updated the /doc for {{find medical sources}} so it looks better now. A "for-later" issue, is that all the "notices" templates (like {{Medical sources notice}}) need to be updated to updated to handle up to four search params, and the /doc to match. But nobody is using these, so it doesn't matter for the cutover.
The only other issue is to write up the launch sequence, as there are a couple of things that have to be done in a certain order, and some module pages to move around or update. The central items for launch are: move the wrapper (currently in Find_sources/sandbox) to Find_sources, and move Talk_header/sandbox to Talk_header (BUT: one line has to be changed before moving, namely the one that calls Find_sources/sandbox; that has to change to Find_sources). I think we should create a smoke test group for the Talk_header/testcases page that we can use to quickly detect any possible problems during and just after the launch. That's a relatively short set of tests that will detect if anything is badly broken, but doesn't test every possible minor thing that all the rest of the testcases are about. I'm going to take a day or two off of this to clear my head, but if you want to write up a smoke test, you don't need any new testcases, just pick and choose from ones that are already there, that are like the "most important" or "most likely to break" from this change, group them together under a new H2 heading right at the top of the testcases page, and call it "Smoke test group" or similar. Include a few very basic tests from non-find_sources stuff, too, just to make sure the launch doesn't break other stuff. Probably no more than a dozen tests, because the whole point of a smoke test is you can check them quickly, and frequently if necessary. Mathglot (talk) 05:41, 1 November 2021 (UTC)
I added a smoke test section to Template:Talk header/testcases. I think this will identify any truly critical bugs. The Template:Find sources/testcases page is easy enough to run through in full. Let me know when you think you'll be ready to do the cutover and I can block off some time to make sure I'm online. - Wikmoz (talk) 20:27, 1 November 2021 (UTC)
THe smoke test section looks good, thanks! Mathglot (talk) 00:15, 3 November 2021 (UTC)
Mathglot, any chance that this can go out on Nov 3? I'd be available to run through all the tests in the afternoon. - Wikmoz (talk) 20:59, 2 November 2021 (UTC)
@Wikmoz: I doubt it, because I still need to come up with the launch sequence (and rollback sequence, just in case). I was thinking about coordinating with you (and Sdkb, and anyone else who is interested) via an IRC channel. I've used these only once or twice, but it's basically a Wikipedia-approved chat platform on Libera chat. I'm not sure if we can create our own "room" just for the purpose, which would be ideal, or whether we have to pick an existing one. Here is the list of m:IRC/Channels; a general overview is at m:IRC. I just registered there as "Mathglot"; command-line help is available if you type /nick YourWikipediaUserid (Enter), then /msg NickServ HELP Register; or go to See also https://libera.chat/guides/registration. If you feel like looking into IRC to see what's what, and whether you think it's a good venue for this, that could help. I asked at #wikimedia-tech which would be the best channel, and someone suggested channel #wikipedia-en but I just tried that, and it's so full I was shunted to an #wikimedia-overflow, so we might want a smaller chat space than that one. Mathglot (talk) 00:11, 3 November 2021 (UTC)
Gotcha. Starting a new talk page section might be the easiest so we can thread issues and link to and demonstrate things right here. Drafting replies in notepad prevents edit conflicts. That said, I'd be happy to jump on IRC. There are also simple services like Talk.chat that let you create chat rooms on the fly without registration. Wikmoz (talk) 03:03, 3 November 2021 (UTC)
Would it make sense to create a temporary talk page at Template talk:Find sources/launch exclusively for the launch steps and cutover-related discussion? In a week, we can merge it back into the main talk page for archiving. - Wikmoz (talk) 18:15, 3 November 2021 (UTC)

Wrapper cutover issue

When everything else looks good, the slightly tricky part is the cutover of the wrapper, and installation of {{find general sources}} as part of the module.

Live template {{find sources}} is currently a 107-byte template that just invokes the module, similar to {{find video game sources}} (and now also {{find medical sources}}; thanks, Wikmoz!). After cutover, {{find sources}} will contain the current content of the wrapper template (currently in the sandbox) which does the autoproject detection and search domain override. But what about the original, 'vanilla' find_sources, where does that live, when the sandbox wrapper is cut over? That functionality is currently in two places: in {{find sources}} (until cutover) and in {{find general sources}} which nobody uses currently, and (except for one autodoc param to distinguish it) is currently a 100+ byte invocation of the Module, and just a duplicate of {{find sources}}. So, the cutover is a bit like a round-robin move, where (the functionality of) current "find sources" moves to "find general sources", and "find sources/sandbox" moves (physically) to "find sources" enabling the new functionality.

Ideally these would happen simultaneously, and I have yet to confirm whether it's possible to do it without a very brief outage (a few minutes) of "find sources" functionality from the Talk header. I think so, by duplicating effort in the module where there are config submodules for both "find sources" and "find general sources", then cutting over the wrapper from /sandbox to "find sources" live, and then deleting all the "find sources" submodules which will no longer be needed. If a brief outage is preferable in order to save having to execute technical deletes, then the "find sources" configs can just be moved to "find general sources", which will leave invocations of "find sources" inoperative across the encyclopedia until the sandbox is moved into place.

There is an alternative approach that I don't recommend, which is to leave the "find sources" config in the module under the current name, and then code the {{find general sources}} template to invoke the module via the "find sources" config. That would work, and would avoid an outage or having to rename or delete module configs. However, imho that would be very confusing, in particular, about the difference between "find sources" in template space and in module space, and why the names are apparently misaligned; I think that would confuse template editors forever after, and probably make it harder to upgrade the templates and modules afterward. So, I don't recommend this approach; separating out the name "find sources" going forward as belonging only to the wrapper which selects which source links domain should be provided, and "find general sources" as just one of several domains, no different from "medical" or "video", will make it far easier imho to upgrade the wrapper, the templates, and the module going forward.

The smoothest approach, with no outage and a seamless transition to new functionality, is to have duplicate submodule configs for "find sources" and "find general sources", cut over the wrapper, then delete the "find sources" configs at one's convenience, and that's the one I would vote for.

This is just a sneak preview to familiarize interested parties with what's coming, and I may have missed something, but that's the basic outline of the wrapper cutover issue, which when completed will finalize the implementation of the new Talk header/find-sources functionality. Mathglot (talk) 19:48, 29 October 2021 (UTC)

Primefac and Trialpears, not sure if you've been lurking, but your thoughts would be appreciated at this point, especially if you see anything I've missed or something above that needs amending. Sdkb and Wikmoz, I know you're watching . We're almost there, guys! Am I just irretrievably nerdy for finding this kind of exciting? Where's that Wikipedaholic test? Mathglot (talk) 20:08, 29 October 2021 (UTC)
I find it exciting, too I haven't been watching as closely as before, since my strength is generally more with ideas than with the technical implementation, but I'm definitely still here and I'll let you know if I notice any issues. Cheers, {{u|Sdkb}} talk 20:43, 29 October 2021 (UTC)

Oh, one bit I forgot: the {{Talk header}} cutover: the {{Talk header/sandbox}} has the new domain code (diff), and also invokes {{find sources/sandbox}} for testing. To cut over the Talk_header template, we need to keep the domain code, and drop the transclusion of the /sandbox version of find_sources. That should make the final transition easier than I thought. Mathglot (talk) 20:21, 29 October 2021 (UTC)

Never done anything like this before so I'll leave it to you and Primefac and Trialpears to figure out. Not sure which approach is safer. In the grand scheme of things, I don't think it's the end of the world if we lose Find sources for a few minutes, especially if there's a clear and quick rollback option. - Wikmoz (talk) 04:20, 30 October 2021 (UTC)
Given the time it takes for the server caches to clear, I doubt anyone will even notice. Primefac (talk) 08:51, 30 October 2021 (UTC)

Launch: 'Find sources' 2021 update

Launch plan and post-cutover items

Launch plan. (Cutover has occurred; follow-up in progress...)

This is the Launch plan for the addition of Dynamic find sources functionality to Template:Talk header:

prep

FIND SOURCES: {{find sources}}
----
TALK HEADER: {{talk header|demospace=1|noarchive=yes}}
----
TALK HEADER (video): {{talk header|demospace=1|noarchive=yes|domain=video}}
TALK HEADER (medical): {{talk header|demospace=1|noarchive=yes|domain=medical}}
TALK HEADER (general): {{talk header|demospace=1|noarchive=yes|domain=general}}
      • Hit [OK]. Note: pre-launch, this will generate red 'Preview warning' messages under each header; this is expected, and should go away post-launch.

launch

Perform the first 3 (or '2' if no config changes) steps as close together as practical:

  1. Module config changes: (t.b.d.; definition in progress; this might be an empty step)
  2. Cutover of find sources/sandbox:
  3. Talk header cutover:
  4. Testing – if testing fails at any step below, stop, notify IRC channel, and proceed to Rollback
    • First post-launch glimpse: go to the ExpandTemplates browser tab, hit [OK] again:
      • all the red 'Preview warnings' should go away
      • the 1st and 3rd Talk header templates should show medical links for "Giardiasis"; the 2nd & 4th 'video' and 'general' links.
    • Run smoke tests for Talk header
    • If smoke tests look good, then test new functionality in Template:Talk header/testcases using tests in sections #E. WikiProject detection and #F. WikiProject override
    • If that passes, go on to remaining tests on the testcases page. (Note that 'green border' in the testcases doesn't indicate a 'pass', it just means that the sandbox and the template produce the same result, buggy or not; after cutover, they *will* produce the same result, so they should all be green.)

rollback

t.b.a.

post-launch

In case of a successful launch, then:

  • Update the /doc page to include Dynamic find sources capability
    • Descriptions of params |domain= and alias |search-domain=
    • Parameters – vertical and horizontal lists. Maybe a rewrite later, as this section does not describe the parameters one by one in order(or maybe group them by function since there are so many: archiving params, find-sources params etc.)
    • Testing issues section – some of the info from testcases regarding in situ or ExpandTemplates could go here, with links to the testcases page.
    • TemplateData section – new param and alias
    • See also section – probably should add the three flavors of find-sources template per search domain
  • Notifications – the following should be notified in the case of a successful cutover:
  • After the dust settles:
    • open a discussion referencing the open or deferred items listed in the #Status summary, section 'Deferred feature backlog'.
    • Decide whether or not to re-merge Template talk:Find sources to Module talk:Find sources;
    • Watch for bug reports/feature requests which normally accompany launch of new functionality.
    • Fix some broken {{tl}} links in sections D or E of the testcases.

Sections on module config and rollback still t.b.a. Mathglot (talk) 03:35, 3 November 2021 (UTC)

@Wikmoz:, what's your timing like, coming up? Mathglot (talk) 19:18, 5 November 2021 (UTC)

I can be available Saturday or Sunday afternoon (US EDT). - Wikmoz (talk) 01:54, 6 November 2021 (UTC)
@Wikmoz: Did you try IRC? Let's try channel #wikipedia-coffeehouse connect tomorrow? Mathglot (talk) 06:22, 6 November 2021 (UTC)
Mathglot, yes. All set up in IRC. Just sent you a message there. - Wikmoz (talk) 17:16, 6 November 2021 (UTC)
@Wikmoz: I'm fairly new to IRC myself, and don't see it. Which channel are you in? Mathglot (talk) 17:23, 6 November 2021 (UTC)
Are you wik-irc? I see you in a wik-irc room, and also in coffeehouse and I said 'hello' there. Mathglot (talk) 17:27, 6 November 2021 (UTC)

Sdkb if you'd like to join us, we're just getting started at IRC channel #wikipedia-coffeehouse connect. Mathglot (talk) 17:36, 6 November 2021 (UTC)

I'm busy this afternoon (only have a sec now), but best wishes! {{u|Sdkb}} talk 17:44, 6 November 2021 (UTC)
Thanks! Mathglot (talk) 17:48, 6 November 2021 (UTC)

Cutover complete

Cutover of dynamic find-sources links with project autodetection and search-domain override completed successfully. Documentation updates, notifications, and other post-launch tasks are coming up next. Mathglot (talk) 20:19, 6 November 2021 (UTC)

Additional piece

There was a missing piece in the immediate post-launch plan, namely, to check for incoming redirect links to {{find sources}} and point them to {{find general sources}}. There were 20 of them (now 19, after the fix for the bug reported below). Here they are: List of remaining redirects. These will have to all be adjusted. Mathglot (talk) 02:47, 7 November 2021 (UTC)

Redirect worksheet for templates pointing to {{Find sources}} to be redirected

Please check the box when a template has been redirected to {{find general sources}}:

Mathglot (talk) 02:55, 7 November 2021 (UTC)

Bug reports

Fullpagename bug: (resolved) – titles with paren disambig included in the search param

Fullpagename bug: A bug in {{find sources}} has been identified by User:Dream Focus at this discussion, concerning the links at Wikipedia:Articles for deletion/House Hippo (2nd nomination). This should search for "House Hippo" but is currently searching for "Articles for deletion/House Hippo (2nd nomination)", i.e., it's taking FULLPAGENAME instead of SUBPAGENAME. Not sure if this is the template or the module config; if I can't figure it out quickly we will have to roll back. User:Wikmoz, are you around to help track this one down? Mathglot (talk) 02:18, 7 November 2021 (UTC)

This appears to be a two-part problem: they are using a redirect template, {{find sources AFD}}, which previously redirected to {{find sources}}; I made a temp fix to redirect it to {{find general sources}}, which may fix their problem, but not the general one. There are two pieces left to fix: we have to check for other redirects that previously redirected to "find sources", and point them to "find general sources". That will fix most problems seen in the wild. But, there's still another piece to add, concerning articles with parenthetical disambig clauses, for which we need to add test cases (and should probably also throw some test cases in for page titles with commas, like Wilmington, Delaware). Lastly, we have to think about what the wrapper should do about the default search for titles with parenthetical disambig; probably just drop them. Mathglot (talk) 02:41, 7 November 2021 (UTC)
I've gone deeper into this, and I don't think we have to analyze the parenthetical disambig cases in the wrapper like I previously thought; the lua code in the Module already handles it (roughly lines 112-126 of the module) and the Module doc refers to it at Module:Find sources#From wikitext: ":Alternatively the following can be used, which will convert any title to a search query, with the main portion of the title in quotation marks, and any parenthetical disambiguator outside quotation marks." with the example, {{#invoke:Find sources|template|title=title}} and that is the key to the bug and how to fix it. I had not noticed that template {{find sources}} handled a param |title=, because I never saw it in the wild, until it broke in the Afd, where they *do* use it, coding the find-sources invocation this way: {{Find sources AFD|title=House Hippo}}. The redirect to {{find general sources}} was a stopgap that fixed their problem, but it won't fix live cases of invocations like {{find sources|title=Main title (terms in parens)}} (if there are any; I haven't seen any). But even if there aren't, it's a legal construct, so the wrapper has to handle it by just passing it through as arg1, and let the module deal with it. We also need new testcases for it. Mathglot (talk) 04:54, 7 November 2021 (UTC)
Great workaround! What's weird is that Template:Talk header/testcases has the same URL structure (without the parens) and works perfectly since this fix. So I'm not quite following how the "Articles for deletion/" part of the URL was getting in there. But it sounds like you have the solution in mind.
It might be good to get a bot to help clean up some of the obsolete needless redirects. Not sure if or how anyone could do the same for unused or little-used parameters, especially with the increasing use of modules, where code bloat can become increasingly expensive (in developer time) as the code base expands.
I got a little sidetracked looking at Wikipedia:WikiProject Deletion sorting/Lists and trying to figure out how it generates the header with {{Find sources AFD}} and the "title=" parameter. - Wikmoz (talk) 06:11, 7 November 2021 (UTC)
Wikmoz, It seems a matter of curiosity how they generate it, but the important thing is that they do (and so, others might, too) so we need to handle it. The sandbox has the new version with an attempted fix, here, but I haven't run any tests yet so no claims about it yet. We can use that to test in ExpandTemplates or in situ. I'll check back tomorrow morning; I've been on this all day long. I'm just glad I was monitoring when they found the bug, or probably everything would've been rolled back by now. Mathglot (talk) 08:46, 7 November 2021 (UTC)
Marking the Fullpagename bug as previously  Resolved and collapsing. There should be some test cases added for the parens or comma case, as noted above. Mathglot (talk) 17:46, 8 January 2022 (UTC)

Paren disambig bug: (resolved) – parenthetical expression double-quoted and shouldn't be

Paren disambig bug: sandbox double-quotes the entire title including parenthetical disambiguation, and doesn't strip the parens; {{find general sources}} does the right thing, stripping parens and removing the disambig clause from the double-quoted part of the query:

  • Execute both in Special:ExpandTemplates with with ContextTitle=Talk:John Smith (anatomist and chemist):
    • {{find general sources}} searches "John Smith" anatomist and chemist -wikipedia which is correct
    • {{find sources/sandbox}} searches "John Smith (anatomist and chemist)" -wikipedia, which isn't.

(Live does the same thing.) Mathglot (talk) 09:31, 7 November 2021 (UTC)

@Wikmoz: okay, I think I've got it in the sandbox, now. Am kinda exhausted, so didn't write up test cases, but try comparing the two cases above, and also repeat them adding another level to the Title (like the Afd thing), so maybe ContextTitle=Wikipedia:Articles for deletion/John Smith (anatomist and chemist). On the way to solving this, I wrote two new templates (which I didn't end up needing in the end, but they helped get me here; you can find them at {{trim parens}} and {{str parendis}}) which maybe might help testing, I dunno. Lmk if you see anything, and if not we can cut this one over, and then (I hope!) we will really be done. (please Reply to icon mention me on reply; thanks!) Mathglot (talk) 00:50, 8 November 2021 (UTC) Turns out that already existed under another name; oh well. Redirected. Mathglot (talk) 00:54, 8 November 2021 (UTC)
If this one looks good to you, we can cut it over and maybe wait a couple of days just to make sure, then we can make the launch announcement on WP:VPR. I'm thinking we should tell WT:MED now, because that would elicit more attention and maybe even some bug reports if there's still any out there, and better hear it from them, than admins at the Interface noticeboard, or the large audience at VPR. Sdkb, also interested in your read on this? Mathglot (talk) 01:06, 8 November 2021 (UTC)
Mathglot: Nice work! This looks good. The standard ones like John Smith (anatomist and chemist) obviously check out. I also checked the most complex ones that I could find, like:
Wikipedia:Articles for deletion/Quicksilver (American game show) (2nd nomination) Not a real use case since the AFD template strips out (2nd nomination)
Wikipedia:Articles for deletion/List of Digimon Adventure (2020 TV series) characters
...just to see if anything blew up. {{find sources/sandbox}} and {{find general sources}} produced the same output so I think we're good. - Wikmoz (talk) 05:17, 8 November 2021 (UTC)
@Wikmoz:, thanks, I thought so too. So, the fix is now released. I'm ready to notify the folks at WP:WikiProject Medicine, unless you can think of any reason we shouldn't yet. Mathglot (talk) 05:25, 8 November 2021 (UTC)
Absolutely! Go for it. - Wikmoz (talk) 05:26, 8 November 2021 (UTC)
Marking the Paren disambig bug as previously  Resolved and collapsing. Mathglot (talk) 17:55, 8 January 2022 (UTC)

Missing redirect detection bug: (resolved) – Project Med redirects like WPMED not handled

Missing redirect detection bug: This is a bug of omission, that manifests on a Talk page like Talk:Lacosamide that belongs to project Medicine and has a TPH (Talk page header), but is showing "general" links instead of medical links. The reason for this is simple: the code on the Talk page uses the redirect Template:WPMED instead of the code {{WikiProject Medicine}}, and the WikiProject autodetection code doesn't recognize that. So, the fix involves ensuring that the autodetection code (currently found in {{Find sources/top proj}}) accounts for this redirect. There are other redirects as well: a total of 15 redirects to project Medicine, but not all of them need to be accounted for in the detection code. Not all of them are used at all, or at least on article Talk pages; for example, Template:CMedWikiProject is another redirect to project Medicine and it is not used on any Talk page, so we don't need to worry about that recognizing that one in the code. Another redirect, Template:WPMEDICINE, has only six transclusions from Talk space and it's not worth writing code for such a small number; it's easier to just retarget the those six articles to use the standard WikiProject name. So we should go through the list of the other 12 redirects, find how many TP's transclude them, and if it's just a handful, retarget them, and if it's a large number, then we'll add it to the list of redirects that need to be detected in the software. Mathglot (talk) 05:00, 9 November 2021 (UTC)

So I checked the redirects to WP:WikiProject Medicine, and counted how many are in use on Talk pages, and six of them aren't used at all, and a few more have just a handful of uses; there are only two that have more than 50. See Template:Talk header/testcases#E-6a (which isn't really the right place to record this information, but will have to do for now). We can just change the Talk pages using the less frequently used redirects so there will only be a few left, which is a lot easier to deal with in the project detection template and keep it comprehensible and upgradable.
We should probably do a similar analysis for WP:WikiProject Video games. Unfortunately, there are 306 redirects to it but I have a feeling that not many are in direct use from Talk pages. For example, for Wikipedia:VG, there are hundreds of links, but no transclusions from Talk space. So, it may not be that bad, but we'll have to look at that later. Mathglot (talk) 10:24, 9 November 2021 (UTC)
Sandbox created for this here, and testcases here. No code yet, but it should be straightforward. Mathglot (talk) 23:54, 9 November 2021 (UTC)
I've upgraded the regex to handle several redirect variants (notably, upper/lower case 'p' and 'm' in 'WikiProject Medicine'). Next up, is to further upgrade it to handle the 'wpmed' suite of redirects, where {{WPMED}} alone has 15,880 articles which transclude it. Mathglot (talk) 10:20, 11 November 2021 (UTC)
Wikmoz, the project autodetect sandbox has been further upgraded to handle all redirects that I am aware of. Sandboxes are chained together so they can be tested at higher levels as needed, starting with:
Additional test cases for */top proj/sandbox are here; I haven't added test cases at the higher levels for Med redirects yet, but any of them can be tested in situ or at Special:ExpandTemplates using the suggested articles in the test cases. In particular, Sandy's question at WT:MED works now, and can be demonstrated at ExpandTemplates by setting the context title to Talk:Multiple sclerosis, and placing {{talk header/sandbox}} in the wikitext input box. If it looks good, launch of the upgraded functionality to handle med redirects involves copying */top proj/sandbox to live, and abandoning the other two sandboxes. Mathglot (talk) 03:00, 22 November 2021 (UTC)
Mathglot, really sorry for the late reply. I tested some of the test cases in Special:ExpandTemplates. {{Find sources/top proj/sandbox}} returns the correct result but {{talk header/sandbox}} does not. I know the initial fix just removes the case sensitivity. Would this be ready to go around the same time as the icon update? - Wikmoz (talk) 19:49, 24 December 2021 (UTC)
@Wikmoz:, that's because the sandbox was resynced to live for some other feature, and therefore no longer contains our changes for project autoredirect. Last I recall, our sandbox version was working, but I hadn't completed all the testing so didn't release, and then I got sidetracked on other things. In the interim, a different feature was added to the sandbox (here) based on the live template (thus, without the unreleased redirect fixes) and then released to live on 27 Nov‎. Presumably they checked all the test cases, but I don't really know, so first we'd have to check if the now live version broke anything in the working project autodect version of 6 Nov., before resyncing the sandbox. (If it broke anything, we'd have to back out their live change.) Before resyncing the sandbox to it, however, we'd have to coordinate with Trialpears who seems to be in the middle of something, or at least, the sandbox has several unreleased changes to it.
I may briefly revert the sandbox to rev. 1056469374‎ of 22 Nov. just to quickly check the smoke test and a few of the proj-autodetect cases, to determine if the 27 Nov. release broke anything, before reverting the sandbox back to its current state so Trialpears can carry on. Mathglot (talk) 20:39, 24 December 2021 (UTC)
@Mathglot sorry if I inconvenienced the project. My changes have now been released and the old sandbox version restored so people can check out how it would look. I'm so looking forward to seeing this released! --Trialpears (talk) 21:12, 25 December 2021 (UTC)
@Trialpears:, no inconvenience; just making sure we're not stepping on each other's toes. There's an obvious alternate approach to this, which is to use sandbox2, sandbox3, and so on, but the limitation on this, is that the {{test case}} suite of templates works on a config that defines the sandbox subpage as 'sandbox', but I think if we created Template:Test case2 which invoked the module with a 'wrapperConfig' using 'sandbox2' then it would work, but I'm a little unclear how 'wrapperConfig' works. I may post a request a the Module talk page about that. Mathglot (talk) 21:46, 25 December 2021 (UTC)
@Wikmoz:, the sandbox has been quiescent for a while, so I synced it to latest template version and reapplied the changes to reenable testing of the missing med redirect bug. I haven't tried it yet, but this should mean the sandbox is now once again picking up the pending changes for missing med redirects, as it was back on 22 November (see this comment) but lost by the time you tried it (this comment). Test cases are included for template {{Find sources/top proj}} in test case B-2, and I'm pretty sure I unit-tested that already back in November for that template in isolation, so now we just have to test Template:Talk header using the in situ or ExpandTemplates technique. This essentially means we have to retest Template:Talk header/testcases#E-2, but instead of using the four articles listed there, using instead the articles listed in test cases B-2a through B-2j listed here. (Just one or two articles each; there's a dozen or more in subtests a, b, and c, but we don't need to do them all. In theory, we should verify before each test that the redirect hasn't been changed in the interim, but that's highly unlikely so probably not needed.) Mathglot (talk) 19:57, 6 January 2022 (UTC)
Mathglot, sorry for the delay. Really great work!! I tested a number of the B2 cases and they all look good with one exception noted below. Several of the E2a test cases (Talk:Dengue, Talk:Psychosurgery, Talk:Pearl_Index) don't include the talk header template. I assume that no change is expected for these topics. Just that they don't break. The one failed case was Talk:Guillain–Barré syndrome. It has an "auto=yes" parameter that breaks in preview mode. - Wikmoz (talk) 22:11, 9 January 2022 (UTC)
@Wikmoz:, for all of the articles in E2a, if I'm doing in situ tests, I just paste the sandbox invocation at the top of the talk page, and then it should show proper results. (And then I cancel out.) For ExpandTemplates, it does what it does, regardless whether the talk page has a header or not.
Thanks for the report on Guillain–Barré. Param |auto= is not one of the parameters of Template:Talk header, so it's proper that the sandbox rejects it (through the invocation of Module:Check for unknown parameters near the bottom of the template). What I don't understand, is why the current state of Talk:Guillain–Barré syndrome fails to run into that same problem live. The transclusion is of the redirect Template:talkheader which should hit that same parameter check path as its target Template:Talk header, but apparently isn't. As "auto" is an unknown parameter, and the sandbox is doing the right thing by flagging it, I'm not overly concerned about it—we can just remove the invalid "auto" param from the transclusion on Talk:G-B, since I consider the sandbox as "working as designed" by flagging the error. Otoh, if there are hundreds of transclusions using that redirect and they use |auto=yes, then it would be annoying for dozens of Talk pages to suddenly appear with on-page errors that weren't there before, even though that's the correct behavior.
User:Qwerfjkl is very knowledgeable about advanced search features including modifiers like hastemplate and others, so I'd like to enlist their assistance here with an advanced search to get an idea of how many pages contain the redirect {{talkheader}} along with the parameter |auto=, like Talk:Guillain–Barré syndrome does. If it's a low number (< 50?), I propose we just edit the pages and remove the invalid param, and then we don't have to look into what's causing the behavior (that is, the omission of the red warning live; although I admit to some curiosity, as I don't understand why it isn't happening now). If there's a lot of occurrences, then we need to find some solution, *even* if the sandbox is working as desired and emitting an appropriate 'unknown parameter' warning from the sandbox because of it. Mathglot (talk) 00:01, 10 January 2022 (UTC)
Did a little more debugging. The error is visible with the current template header in preview mode. Similarly, when published with the sandbox talk header in this revision, the error doesn't display on live. So I think it's safe to ignore. Sorry, I should have done those checks with the initial report. - Wikmoz (talk) 03:45, 10 January 2022 (UTC)
@Mathglot: As requested (9 results). Qwerfjkltalk 07:13, 10 January 2022 (UTC)
@Qwerfjkl:, very cool, thanks! Wikmoz, given this situation, I think we can just remove all the occurrences of it, and then if there are no other difficulties lurking, we can release the fix for the "Missing redirect detection bug". Mathglot (talk) 07:20, 10 January 2022 (UTC)
I removed those cases, and a few more I found that didn't show up in that search. Also found 44 pointless cases of |search=yes that ought to be dropped, because it's the default and doesn't do anything (but doesn't hurt, either, so I left them). Mathglot (talk) 09:36, 12 January 2022 (UTC)
I beefed up the E-2 test cases, and I've gone through the E-2a tests at Template:Talk header/testcases#E-2 involving 18 articles listed there, and they all pass. If you don't find anything, I think we're ready to release it. Reminder: though it affects template {{Talk header}}, the locus of the fix is in Template:Find sources/top proj/sandbox, and that is the only page that needs to be moved after testing is complete; there is also new template {{Find sources/proj/is med}} but as it is new didn't require a sandbox version, it's already "released" so to speak, but none of the existing live templates are aware of it, and won't be, until top_proj is updated. Mathglot (talk) 06:40, 7 January 2022 (UTC)
(Marking the Missing redirect detection bug as in progress, and attaching an open collapse bar.)Mathglot (talk) 18:10, 8 January 2022 (UTC)

Copied the sandbox of top_proj, which implements the fix to the missing redirect detection bug. Restested the 18 articles listed at the bottom of test E-2a after launch, and all of them passed live. Declaring this bug  Resolved as of edit 1065379198. Mathglot (talk) 07:45, 13 January 2022 (UTC)

Marking the Missing redirect detection bug as  Resolved and collapsing. Mathglot (talk) 07:50, 13 January 2022 (UTC)

Launch announcement and feedback

I have notified WikiProject Medicine in the first public announcement about the launch, and have invited them to leave feedback about bugs, feature requests, and general questions here, or at WT:MED. Thanks, Mathglot (talk) 08:14, 8 November 2021 (UTC)

Forgot to add that I notified WP:WikiProject Video games later in November, and the notification got a bit of feedback, then fell silent and got archived. Presently, it can be found here. Mathglot (talk) 04:20, 24 December 2021 (UTC)

Enhancements and deferred features

Stacking config

Have started thinking about enhancements and deferred features, even though we're not quite ready for it yet. With respect to stacked lists, I was thinking about an additional kind of configurable option, which would define something a bit more fine-grained than the current brute force, top-to-bottom approach of "med first, then video, then general", which may feel even more brutish when we add more domains, like biography or military. We could have a stacking config (probably just a subtemplate) that would define how to handle multiple detected projects. E.g., 1. med + bio ⟶ stack both; 2. bio + mil ⟶ both; vg + med ⟶ med only; vg + bio ⟶ both; and so on. A refinement could be, specify which one is stacked on top (or what order, if 3 or more), irrespective of the order of WikiProjects found. Mathglot (talk) 01:48, 8 November 2021 (UTC)

Not sure if we ever quantified the number of topics that would benefit from this feature but at least with the current domain coverage, the number is going to be incredibly small. After witnessing the complexity involved in accounting for little-used parameters in other templates, I'd favor sacrificing flexibility for simplicity in the solution. Also, since we don't know usage numbers, the user benefit (of having multiple lists rather than defaulting to the priority domain or general list) could be infinitesimal. - Wikmoz (talk) 20:48, 8 November 2021 (UTC)
You're right; shelved. Mathglot (talk) 08:48, 9 November 2021 (UTC)

Template icon

Most talk page headers include a left-aligned icon. Currently, 'Find sources' is very helpful but also very easy to miss. Adding a small left-aligned icon would go a long way to ensuring editors find the links. I was picturing a magnifying glass (to suggest "search") in the foreground (for consistency across all 'Find sources' templates) and a domain-specific icon (rod of asclepius, video game controller, globe or stack of books) in the background.

In terms of layout, we could use a smaller icon like the one used here or here so the icon doesn't affect the template height. As an aside, it would be helpful if all templates used in Talk header had their icons standardized in terms of position and size but that's a project for another day.

Icons8 allows for free use of their icons in open source projects. Just searching within their "Color" theme icon collection, there are some great options. There's an overlay option that lets you see the following icons with a search icon overlaid.

- Wikmoz (talk) 21:23, 8 November 2021 (UTC)

I agree, and until a day or two ago, the "notice" (boxed) versions of the three flavors had icons on the left, but they were removed in order to match the look and feel of the integrated "find sources" box generated by the template, which starts with the words "find * sources" on the left, with no icon. {{General sources notice}} and {{Medical sources notice}}, used the default image displayed by a |type=notice tmbox, c:File:Information icon4.svg: . Would be better if we had meaningful icons, which could then be used to replace the words "find *sources" which take up too much horizontal room, maybe keeping them as a tooltip when hovering over the icon. Commons may have some things to look at, such as these:                                 . If they are open source, like the ones you mentioned, then we could ask the folks at Wikipedia:Graphics Lab/Illustration workshop to create some icons in Commons for us based on the ones you found, or even just based on a text description of what you want, such as, "magnifying glass superimposed over bottom-left third of image of rod of asclepius", or some such. Mathglot (talk) 20:28, 9 November 2021 (UTC)

for medical sources

Thanks! Wasn't sure where to request illustrations. I kicked off a discussion here. Not sure whether it's better to use a small search icon (indicating the search action) in the foreground or a large open book (indicating research) in the background as the consistent element (with smaller subject-specific icons in the top third). - Wikmoz (talk) 20:49, 9 November 2021 (UTC)
I'm following that discussion. I left a brief response there, but am splitting part of it here, because this is about how a mockup might look using one of those icons, and that's not really pertinent to the Graphics Lab. Anyway, here's a possible look for how this might eventually look as part of Talk header template at the top of Talk:Giardiasis:
Which I think is not too bad. You might notice that the icon is linked to a Wikipedia Library search (same results as you get by clicking the last link) which, since the resolution of T294919 is far more useful now. Mathglot (talk) 07:55, 12 November 2021 (UTC)
Looks pretty good with the updated icon and a link to MEDRS. :) - Wikmoz (talk) 02:06, 18 November 2021 (UTC)
@Wikmoz: That really does look really good. I'm not sure if this is something that needs to go through an Rfc just to change the icon; it borrows the Asclepius rod from the MED project, so I assume they'd be on board, and the foregrounded loupe just makes it seem so clear what that icon means. Maybe we should just be bold and switch the icon, and see if anyone complains? Mathglot (talk) 04:15, 24 December 2021 (UTC)
I definitely don't think this needs an RFC but it may be worth giving WT:MED a heads up that the Find sources templates will soon display an icon. I can make that post tomorrow. - Wikmoz (talk) 05:15, 24 December 2021 (UTC)
Mathglot, There were no objections on WT:MED. I think we're good to go. - Wikmoz (talk) 06:01, 31 December 2021 (UTC)
 Done. (Also added an H4 section header above, to set off the part of the discussion about the medical sources icon.) Mathglot (talk) 06:47, 31 December 2021 (UTC)

for other sources

@Wikmoz:, now that medical is done (probably the hardest one) would your designer friend (mentioned in this Graphics Lab discussion) be able to do something similar for the icons for the 'general' and 'video games' icons in the smaller size and trying the eclipse gap thing?

Also, I'd like to think about {{Biographical sources notice}} as well, which exists as a standalone template, and uses the Module code; that is, even though it's not currently interpreted by the wrapper, it's still available as a stand-alone. We haven't picked an image yet, and we'd be starting from scratch, but we could do another Graphics Lab request, leveraging the previous one that already has the loupe. There's something about the commons image with the book and the fountain pen that makes me think "biography" (maybe I'm thinking of Boswell, laboring over Life of Johnson by candlelight or something), but maybe you can find something better? Here are some more icon ideas online. Mathglot (talk) 06:59, 31 December 2021 (UTC)

Nice! I'll check out the biography options. For the others, you weren't a fan of these? There's an eclipse gap in there... easy to see at higher resolutions but could be made wider.
- Wikmoz (talk) 22:28, 31 December 2021 (UTC)
Yes, I am a fan of those, I just copied the images of the GLab discussion, I guess those weren't the right ones. Please use yours. Mathglot (talk) 12:54, 1 January 2022 (UTC)
I will try to get a biography icon now. Regarding the medical icon and others, I see that the icon appears in Template:Medical sources notice but doesn't flow through to the Talk header template. Could that be the next project after the project medicine redirects fix? - Wikmoz (talk) 22:15, 9 January 2022 (UTC)
I'd like to see it, but I don't want to wear out our welcome, so I'm just wondering if we should pause on icons (because that is such a visible change), and see whether we should proceed with biography sources in the Talk header, or not. But yeah, I think the icons would be a great, visual indicator of the link set. Mathglot (talk) 09:58, 12 January 2022 (UTC)
Mathglot, I added an option for biography search to the set above.
Given the feedback on WT:MED, it seemed like it would be safe to add the icons. Right now, the whole section just seems too easy to miss. Maybe we can limit the initial rollout to the topic-specific searches? - Wikmoz (talk) 22:54, 20 January 2022 (UTC)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Template_talk:Find_sources/Archive_2&oldid=1197704042"