Wikipedia talk:Identifying reliable sources (medicine)/Archive 7

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia
Archive 1 ← Archive 5 Archive 6 Archive 7 Archive 8 Archive 9 Archive 10

toxicity

Here is an article that is killing me. Bisphenol A. There is (to me) an insane laundry list of content - a lot of it supported solely by primary studies - in the section called "health effects". It has to do with people wanting to really ring alarm bells on the potential toxicity of this stuff. I think it is hard to deal with tox under MEDRS. I am getting lots of pushback against killing the content that is based solely on primary sources. I think if content can't be supported by 2ndary sources and is in a section called Health Effects, it's gotta go. There are very different worldviews of folks involved, on what kind of sourcing this kind of content requires. Toxicity, mind you. Very interested to hear what folks have to say. A few of the frequent commenters here have dropped in on the conversation there, on talk, but I am interested in hearing the wider community too. What do you think? Jytdog (talk) 08:02, 8 April 2013 (UTC) (did a little copyediting to fix typos/errors; added wikilink to the Talk page Jytdog (talk) 15:05, 8 April 2013 (UTC))

My personal experience: I went to that article since I recently had a son and everybody talks about bisphenol. I found it so laundry list that I simply decided that it was useless sh**. Unless it is completely reformatted its usefulness is similar to writing bisphenol in the google search or pubmed. Most of its content is cherry-picked crap... However unless somebody is willing to spend maaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaany hours improving it, it would be hard to make any changes being it so controversial. --Garrondo (talk) 14:34, 8 April 2013 (UTC)
Article watchlisted. Jytdog, I agree the primary-source based content needs to be removed. BPA is well-studied and there's plenty of secondary sources available. After a very basic search here are three sources to consider that are not currently used in the article: PMID 22889897, PMID 22481365, PMID 22360935. No reason for primary studies on this topic in the article. Zad68 14:46, 8 April 2013 (UTC)
Thanks, both of you. You can see on the talk page and history that I made an initial bold attempt to remove content based on primary sources a month or two ago, which was too bold and created some bad feelings among editors who had been working on the page. Some of the pushback is about that. Some of it. Just wanted to acknowledge that. But there are philosophical differences. As I tried to emphasize above, I think toxicity is a very hard topic under MEDRS. Tox is based on in vitro and animal studies, which any basic researcher can do. And as Colin mentioned above, it is very worthwhile for basic researchers to draw big conclusions from their work to get more funding and fame. When the subject is toxicity of a product like this, you get lots of basic researchers with no real understanding of what toxicologists actually do, making all kinds of health claims, in primary articles as well as in reviews. I am very interested in this article as a great example, but there is the bigger picture (look at articles about pesticides, etc, too) - toxicology is a field where MEDRS is not as helpful as it could be. It would be great if it said something like "Sources that are most appropriate for content that summarizes results of in vitro or animal studies of the toxicity of products, are secondary and tertiary sources written by toxicologists. Primary studies should not be used to support content that summarizes results of in vitro or animal studies of toxicity. Generate content that summarizes results of in vitro or animal studies of toxicity using secondary and tertiary sources written by non-toxicologists with caution." Something like that. Jytdog (talk) 15:05, 8 April 2013 (UTC) (replaced words "content about toxicity" with "content that summarizes results of in vitro or animal studies of toxicity" Jytdog (talk) 15:18, 8 April 2013 (UTC))
Toxicity isn't that hard under MEDRS. You just need to find a review of the animal studies. Notice that "secondary" doesn't say anything at all about "in-human testing". If you go off to PubMed and put bisphenol a toxicity into the search box, and then click the "Review" button in the left-hand column, and then the "5 years" button to limit it to recent results, you'll get dozens of perfectly good secondary sources. More than 20 have the full text available free online. Use them. WhatamIdoing (talk) 05:19, 9 April 2013 (UTC)
Hi Whatamidoing, thanks for your comment, but I would love it if you responded to what I wrote above. Of course "in human" studies are not relevant for tox - the obvious reason being that it is not ethical to test nonbeneficial substances in humans. What I wrote, is that because tox studies are done in vitro and in animal models, you get all kinds of basic researchers weighing in toxicity, but doing so outside of the way that toxicologists practice their discipline....To elaborate -- they report on experiments where they inject unrealistically large amounts of the substance directly into lab animals (!) or dump large amounts onto cells and say "look look! it kills cells/mice! blah blah blah" in their primary publications, and then other non-toxicologist basic researchers pick up those studies and amplify them, uncritically, in reviews. This is very problematic. If you look at any of the sources cited in the "expert panel" section of the Bisphenol A article, in detail, you will see that (a) they find most of the studies they review to be unusable, and (b) the conclusions that the panels draw are much more tentative than what basic researchers say in reviews. We should not be able to just "use them". MEDRS as currently stated does not allow editors to perform any kind of peer review-type assessment, and rightly so. But that leaves wikipedia in the position of having all kinds of inappropriate content about toxicity. This is why I think it would be very useful to have MEDRS be more explicit about what kind of sources are appropriate for toxicology content. Jytdog (talk) 11:29, 9 April 2013 (UTC)
DUE weight for toxicity, as for anything else, includes what everyone says about toxicity, not just what the right kind of people say about it. Therefore we use all the (secondary) sources, not just the ones that we personally believe are taking the correct approach. WhatamIdoing (talk) 00:03, 10 April 2013 (UTC)
That is a surprising argument! You surely don't mean "everyone". MEDRS is all about basing content on what "the right kind of people" write. I am making an argument that toxicology is a strange bird in the biomedical field and we should have more strict guidelines - I would appreciate it very much if you would address what I wrote above. Thanks Jytdog (talk) 01:51, 10 April 2013 (UTC)
Adding, just want to point out this ref: http://www.ncbi.nlm.nih.gov/pubmed/19931376 which is an article called "Risk to all or none? A comparative analysis of controversies in the health risk assessment of Bisphenol A." from the journal, Reproductive Toxicology, the abstract of which says: "Bisphenol A (BPA) is an endocrine disruptor for which health risk assessment has proven controversial. Conclusions regarding health risks of BPA vary between assessments from "there is no risk to any part of the population" to "there is risk to the entire population". We have carried out a literature study investigating what might be the scientific and/or policy-related reasons for these differences. Ten risk assessments for BPA were scrutinized and several factors were compared between assessments, including estimations of exposure levels, identification of critical study and NOAEL, assessment factors and significance attributed to reports of low-dose effects. Differences in conclusions were mainly influenced by the evaluation of low-dose effects and the uncertainties surrounding the significance of these data for health risk assessment. The results illustrate the impact of differences in risk assessment policy and expert judgment on the risk assessment process and highlight the importance of transparency in this process." The point being, expertise in toxicology matters. Jytdog (talk) 02:05, 10 April 2013 (UTC)
On Wikipedia, we also represent the views of all the major reliable sources, especially when these "non-experts" are scientists who are experts in highly relevant fields. A toxicologist is not automatically a more appropriate expert than a neurologist when the question is what effects a toxin has on nerves. WhatamIdoing (talk) 5:54, 11 April 2013 (UTC)
I am sorry but this is exactly my point. The only reason to include studies about "effects on nerves" is to convey information about health risks. Right? A neurologist (the clinician who takes care of nervous system conditions) may have a vague understanding about the biochemistry/cellular biology at play in the in vitro and animal work, and is unlikely to know how to think about exposure levels and how that translates into "dosing" -- and is unlikely to be the author of any primary study in any case. A neuroscientist (the basic researcher) will likely understand the in vitro work and animal work, but again is not trained, nor experienced in, how to think meaningfully about what those results mean to a typical person nor at what exposure level - about the health effect. But this is what toxicologists are trained to do and spend their careers doing - thinking about how to translate results from in vitro and animal studies into effects on humans, at the exposure level that is relevant to the real world. And (very importantly!) they think about how to design the experiments so that the results are use-able for that purpose. Again if you look at the sources cited in the Bisphenol_a#Expert_panel_conclusions section - which are something like 300 pages long -- you will see that they find most of the primary studies useless for trying to determine risks of health effects -- they were not designed in such a way to be useful for toxicologists. And this is the enormous problem with including all these primary studies, and with including reviews of them by people who are not toxicologists. The result is wikipedia communicating information about health that is scary and may be wrong -- exactly what MEDRS was designed to avoid. Jytdog (talk) 16:42, 11 April 2013 (UTC)
And the toxicologist might know next to nothing about nerves. We do not privilege one specialist over another, especially when multiple specialties are relevant. We don't tell the nerve specialist, "No, no, you are too ignorant to write about toxicology of nerves", and we don't tell the toxin specialist, "No, no, you are too ignorant to write about the toxicology of nerves". We accept both of them. WhatamIdoing (talk) 18:28, 12 April 2013 (UTC)
I'd pay more attention to what a toxicologist has to say about toxicity than what any other specialist has to say on the topic - all else (quality of peer-review, standing within the profession, etc.) being equal. --Anthonyhcole (talk · contribs · email) 18:39, 12 April 2013 (UTC)

No, we should indeed not tell a nerve specialist anything. But yes, yes we should source content from the experts. You would not go to a neurologist for a haircut, nor for open heart surgery. If there is no source by a toxicologist on the effect of a substance on the nervous system, then we should say nothing. I don't understand why you are not responding to my point! Planning in vitro/animal studies so they are useful for toxicological judgement is a discipline, as is interpreting the results for the purposes of toxicology. You are ignoring this. Have you looked at the source I keep pointing you to, to see how toxicologists treat most of the primary studies they analyze? Here, I will put a direct link here for you. http://ntp.niehs.nih.gov/ntp/ohat/bisphenol/bisphenol.pdf -- I will quote from it:

3.0 DEVELOPMENTAL TOXICITY DATA The Panel attended to multiple design and analysis characteristics in judging the acceptability of individual studies. It was our consensus that for a study to be acceptable for this review process, several conditions had to be met. First, effects related to litter of origin needed to be accounted for in design and statistical procedures. Second, animals needed to be dosed via the dam or directly under individual housing conditions. Concern that multiple exposures within a cage to different animals could cause cross-animal contamination across cage-mates led to the determination that this design was not acceptable. Third, a minimum of 6 animals pertreatment condition needed to be used to provide minimal confidence in results. Fourth, if similar tests were conducted at multiple ages, the statistical analyses needed to account for repeated measurement in order not to inflate degrees of freedom. The Panel carefully con­sidered the merits of each study according to these primary criteria, and the related design characteristics represent the most common reasons for judging a study to be unacceptable for our review process. Our intent was to have our review depend most heavily on studies that would have reduced risks for false negative or false positive findings. In addition, the Panel carefully considered the value of studies where bisphenol A was administered anywhere other than to the mouth or stomach of the experimental animal. Human exposure is overwhelmingly oral, and oral exposure produces an internal metabolite profile which is overwhelmingly dominated by the (inactive) glucuronide in both rats and humans. Subcutaneous or parenteral injections result in blood levels of active parent compound which are much higher than those seen after oral exposure. In light of these pharmacoki­netic differences, the Panel concluded that injection studies, unless they proved otherwise, would produce irrelevantly high internal doses of the active parent compound, and would tend to produce ‘‘false positive’’ effects from the point of view of the human oral situation. Thus, the Panel viewed those otherwise adequate studies that injected bisphenol A as providing ‘‘supplemental’’ information (i.e., of limited utility), unless they also analyzed the levels of parent compound and metabolites after the injection. The intent of this approach is limit the impact of those studies which produced an unrealistic and irrelevant internal metabo­lite profile (i.e., one which is significantly different from that experienced by humans). Thus, the closer any given study came to replicating the human situation, the more weight it had in the final analysis" (pp 235-236)

So if you look at the Bisphenol A article, you will find there a study done in monkeys in the "Neurological issues" section. (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2544599/) Here is the methods section:

Animals: Young adult female African green monkeys (Chlorocebus aethiops sabaeus) of reproductive age were used (n = 12; body weight, 4–5 kg).

Surgery and Hormone Treatment. All monkeys were anesthetized (20 mg/kg ketamine, i.m, followed by 20 mg/kg pentobarbital i.v.), intubated, and ovariectomized through a median laparatomy under sterile conditions. In the same surgical session, animals received the following treatments:

Vehicle-treated controls (three animals): a cholesterol-filled 4-cm long Silastic capsule (Dow Corning; 3.5 mm i.d., 4.65 mm o.d.) and a vehicle-filled minipump (Alzet 2ML4 osmotic pump, delivering fluid at a rate of 2.5 μl/h for 4 weeks). EB-treated group (three animals): a Silastic capsule containing crystalline EB and an Alzet minipump loaded with vehicle. BPA-treated group (three animals): a cholesterol-filled Silastic capsule and a BPA-filled Alzet minipump. EB+BPA-treated group (three animals): a Silastic capsule containing crystalline EB and an Alzet minipump loaded with BPA.

We demonstrated that this type of estradiol treatment induces marked changes in CA1 spine synapse density in female monkeys (31). The Alzet minipumps delivering BPA were filled with 4.17 μg/μl of BPA dissolved in propylene glycol (Sigma) to supply BPA at a rate of 50 μg/kg/day. The Silastic capsules and minipumps were implanted below the skin of the back. Minipump performance was verified by measuring the volume of fluid residue extracted from the pumps after the animals were euthanized. After surgery and recovery, the animals were housed in individual cages. They were given buprenorphine analgesia (Buprenex, 0.01 mg/kg i.m.) at the conclusion of surgery, followed by carprofen (Remidil, 2 mg/kg by mouth) every 6 h for the first 2 postoperative days. Water and food intake (soybean-free diet: TD.06476; Harlan Teklad) and incision sites were monitored until complete wound healing occurred.

So these people tore up these monkeys to do a study that does not reflect human exposure - that toxicologists would just throw out, as administering BPA via these implanted minipumps is the same as subQ injection (just a steady dose instead of a bolus). And we cite it. How is this is a good thing, on any level? (i don't want to bring in the whole animal research ethics thing, but how the heck was this approved by their IACUC?) More importantly, why in the world we would cite this as relevant to humans? Jytdog (talk) 04:32, 13 April 2013 (UTC)

I wouldn't have used that study to begin with: it's a primary source, and we want secondary sources. The identity or specialty of the authors is irrelevant, and Wikipedia editors aren't supposed to be evaluating methodology.
The source is a reliable source for the material that it's supporting, which is basically "This particular study came to the following conclusion".
And, again, if you want to improve that section, then you need to find several secondary sources, and add good information. Improvements are not handled by removing everything that you disagree with or whose authors didn't have the exact specialty that you prefer. WhatamIdoing (talk) 21:28, 13 April 2013 (UTC)
Dead horse, whatamidoing. I already apologized for having been too bold and removed content supported only by primary studies in the article. Jytdog (talk) 12:24, 19 April 2013 (UTC)

Alternative medicine

Many, many editors mentioned that alternative medicine articles are having really difficult times on the Wikipedia. IMHO, that should be addressed. One thing, we don't want unproven claims. The other thing, that alternative medicine is really lacking in research comparing to mainstream medicine. That is in fact why we call it "alternative medicine". I think MEDRS shall include a special guideline paragraph that will be applicable to alternative medicine only and will define how unproven but known claims may be stated.

A good example could be D-mannose. In the past years it has gained tremendous popularity in treating bladder infections due it's ability to eliminate e-coli bacteria without killing them as anti-biotics do. There are thousands of anecdotal reports of tremendous success. It has got more than 200 positive 4-5 stars reviews on the amazon alone. Many, many readers will click on d-mannose because they would be interested to read about it in perspective of using it for the bladder infections. Yet, they will find nothing in the article regarding it. Ryanspir (talk) 15:56, 31 March 2013 (UTC)

It's an interesting issue. For example, WebMD has an article about its use as a treatment.[1]. Seems like they handle it well. I can see the rationale for exclusively using peer-reviewed secondary sources, but that practice does leave some gaps in WP's coverage. TimidGuy (talk) 16:13, 31 March 2013 (UTC)
Ryanspir wrote elsewhere: "My general idea is that alternative medicine articles shall be allowed to state the claims for different substances with attribution and remarks in case these claims hasn't been proven."
Yes, that is allowed, if the wording is framed properly (making it clear it's an unproven claim) and the source isn't a commercial one. We need to avoid promotion and selling here, but documenting claims is proper. It still comes down to a question of notability for the article itself. Our WP:Fringe theories guideline applies here.
Otherwise I support a section devoted solely to dealing with alternative medicine issues. In fact, it would be great if we had a guideline devoted to AM, where the relevant aspects of our various policies and guidelines were collected in one place and interpreted in light of how to use them for such issues. The nature of the subject means that we are constantly butting up against the edges of fringe, notability, promotion, medicine, science, research, transitions, politics, culture, sourcing, scams, quackery, pseudoscience, conflict between true believers and science, etc.. -- Brangifer (talk) 16:21, 31 March 2013 (UTC)
TimidGuy's concern about leaving "gaps in WP's coverage" is very legitimate. Our main goal at Wikipedia is to document "the sum total of human knowledge." Once notability has been established, we must fulfill that goal, and we shouldn't have gaps about such subjects. -- Brangifer (talk) 16:40, 31 March 2013 (UTC)
I do not understand the comment "unproven but known claims"? Do you mean to state something like "substance X is used for Y but evidence does not support this indication"? I would be happy with that to a limited extent within the main articles with more in the subarticles. WebMD is not what we want to try to emulate. Their funding model involves taking money from advertisers and than writing articles that are pleasing to those paying the billings.[2] Doc James (talk · contribs · email) (if I write on your page reply on mine) 19:15, 31 March 2013 (UTC)
BTW, do we have any mention of WebMD as an unreliable source? Maybe we should. -- Brangifer (talk) 23:22, 31 March 2013 (UTC)
Agree we should state the WebMD is not a reliable source. Doc James (talk · contribs · email) (if I write on your page reply on mine) 23:39, 31 March 2013 (UTC)
Ryanspir... Here are page hits for d-mannose http://stats.grok.se/en/201303/d-mannose i don't know if an average of 6 hits a day is "many many". And IMO "alternative medicine" treatments that are not proven to be safe and effective are not medicine; and on the flipside, "alternative medicine" that has been proven safe and effective is medicine - the term is pretty useless, actually. Wikipedia presents the scientific consensus in scientific article, and the medical consensus on health articles. And where evidence is lacking, science and medicine say that, or say nothing. So too should wikipedia. The dietary supplement industry is really aggressive in trying to convince people that their products are useful for treating and preventing disease, and many people unfortunately believe them, even though sometimes they turn out to actually be harmful. So I vote for the status quo -- any health claim needs MEDRS sources. If that means we don't sing the tune the dietary supplement industry wants, that is OK with me. This was actually what got me started editing Wikpedia - I really resent the BS that flies around "alternative medicine." Jytdog (talk) 22:00, 31 March 2013 (UTC)
I've seen a lot of people whingeing recently about the lack of evidence for AltMed, and I'm just not buying it. It's not true. PMID 18950249 is a recent review article that covers the use of mannose to treat UTIs. The full text is free online. A search on "d-mannose urinary" gave me more than a thousand hits at PubMed. Three dozen are reviews. What's the problem here? Didn't you even look before just assuming that nothing exists? WhatamIdoing (talk) 00:44, 1 April 2013 (UTC)
ummm, if you actually look at the article, d-mannose gets a column on page 237 that discusses in vitro and mouse work - no human clinical studies upon which to say whether it is safe and effective. Which is what I was saying. Your tone is not appreciated, btw.Jytdog (talk) 01:36, 1 April 2013 (UTC)
Yes: half of page 237 is about d-Mannose and it's summarized on page 240 in the table. We require secondary sources on the topic. The review is a secondary source, so we've met the requirement. We don't actually require human studies or regulatory approval (the meaning of "safe and effective": nothing is "safe and effective" unless and until the FDA or its equivalent says so).
(My comment was not in reply to yours, BTW.) WhatamIdoing (talk) 22:56, 1 April 2013 (UTC)
Oh, thank you, Please pardon my ego and oversensitivity.Jytdog (talk) 00:03, 2 April 2013 (UTC)
question for you, whatamidoing. if I were working on the d-mannose article, and if i were using this article, I would write something like the following: "In vitro and animal studies have shown that bacteria treated with d-mannose do not adhere well to the cells that line the urinary tract. There have been no clinical studies in humans, and the FDA has not approved d-mannose to be marketed as a treatment for UTI." I didn't say this earlier, but I would push back against the source you brought; the journal is called "Alternative Medicine Review"; this article was written by its editor-in-chief, and she works for a company that sells supplements, etc. Pretty COI-y. And the article ends with what I see as a statement unsupportable by the evidence presented in the article, namely: "Botanicals and botanical extracts that can be particularly effective for acute use, but not intended for long-term use, include berberine and uva ursi; whereas, cranberry, mannose, probiotics, and estriol are suitable for long-term prevention." She is telling people to use mannose to prevent UTIs, saying that it is "suitable". This is exactly the kind of stuff that kills me. whatamidoing - would you really use this as a source under MEDRS, and what kind of content would you generate from this source? Jytdog (talk) 00:03, 2 April 2013 (UTC)
I would use a source like this to make the plain statement that it is actually being used this way. I would not use a source like this to make a claim that it works, or that it should be used this way.
Per WP:MEDASSESS, I would ignore the author's funding, i.e., who signs her paycheck. Sources cannot violate WP:COI, which is entirely concerned with the behavior of Wikipedians. They might or might not be WP:Independent sources, but they cannot have a COI problem as far as Wikipedia is concerned. It is, of course, perfectly usual for a journal's editors to submit publications to the journal they edit; that's what "peer" review means. One of the purposes of blinded reviews is to prevent reviewers from giving editors a less strict review. WhatamIdoing (talk) 16:18, 2 April 2013 (UTC)
if a section on altmed is created, maybe it would be helpful to mandate (as far as a guideline can) some kind of sectioning, along the lines of "How X is used" that simply describes that... (content like: "D-mannose supplements are taken orally (?) by some women to treat or prevent UTIs" and in which it is forbidden to make claims about X itself, like "D-mannose is safe and effective for treating or preventing UTIs") with a following section called something like "Clinical evidence of safety and efficacy" and another section called "Preclinical research", with instruction that content in all 3 sections must use secondary or tertiary MEDRS sources only. In other words, open the door to this content, but provide both structure and higher standards to help deal with fringe, pseudoscience etc... Jytdog (talk) 03:45, 1 April 2013 (UTC)
Seems like a good suggestion Jydog. By the way, here's what MEDRS says about WebMD: " Peer reviewed medical information resources such as WebMD, UpToDate, Mayo Clinic, and eMedicine are usually acceptable sources in themselves, and can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should cite the more established literature directly." TimidGuy (talk) 09:59, 1 April 2013 (UTC)

I taken strong issue with Brangifer's quote "the sum total of human knowledge". That is marketing bullshit by Jimbo and of zero concern to us wrt policy of guideline. It is easily demonstrably false and unhelpful. WP:NOT is one of the longer policy pages. We're here to create an encyclopaedia, which if it is to be at all helpful to a 21st-century reader with a desire for knowledge and wisdom, has to edited in the full meaning of that word.

Wrt d-mannose, I don't support a creation of an "Alt Med" section any more than "In popular culture" sections are a good idea. It is just a flag to say "put your low quality crap here". Just because a substance is used anecdotally or is promoted by shady internet sites doesn't make it alt-med. Cough medicine isn't alt med yet isn't supported by evidence. There are lots of things we do or take with no evidence. There are a few facts one can say about the pills that probably amount to a sentence or two. Probably something neutral like "Urinary tract infections" as a section title is reasonable.

The issue with WebMD, UpToDate, Mayo Clinic, eMedicine, etc are that they are moderate quality sources that are perfectly fine for everyday uncontroversial facts. If someone adds such a fact and cites one of those sites (rather than something truly awful like the Daily Mail) then that's no grounds to delete the edit. But in any area of controversy, we need stronger quality sources. Extraordinary claims demands extraordinary evidence and all that. Editors who spend their lives editing controversial articles may forget there's the rest of the encyclopaedia too. Colin°Talk 14:50, 1 April 2013 (UTC)

You are free to have your opinion about the overarching goal of Wikipedia, but that's what it is. Your opinion is only your opinion, and it happens to be against the main goal of Wikipedia. There is no conflict between that goal and our policies and guidelines. When they are applied properly, we can achieve that goal while maintaining the integrity and usefulness of the encyclopedia. The two complement each other just fine. When in doubt about how to apply policy, it's good to keep that goal in mind and weigh whether one's edits build or diminish the project. It's a fine balance between inclusionism and maintaining the integrity of the encyclopedia. -- Brangifer (talk) 16:11, 1 April 2013 (UTC)
No, the overarching goal of Wikipedia is to be "a multilingual, web-based, free-content encyclopedia". I'm really surprised when I see otherwise rational wikipedians quoting Jimbo's soundbite. It sounds dynamic I suppose, but if it were even partly true, then WP:NOT would be very short indeed. Colin°Talk 16:28, 1 April 2013 (UTC)
Based on secondary sources saying that d-Mannose is actually being used for UTIs, I don't have a problem with repeating this undisputed fact at Mannose, in a section called ==Medical uses== (suggested by WP:MEDMOS#Sections) or even just ==Uses==. Ideally the statement would be something like "Despite having no scientific evidence that it works in humans, d-mannose is used as a dietary supplement by some people for UTIs". (I might also consider including PMID 14631566's safety concerns about high concentrations during pregnancy.) Since we don't have evidence in humans, I'd be much less inclined to mention it at Urinary tract infection. WhatamIdoing (talk) 23:54, 1 April 2013 (UTC)
Hooton TM (PMID 22417256) is succinct:
"Table 4. Strategies for Nonantimicrobial Prevention of Recurrent Acute Uncomplicated Cystitis."
Strategy:"Adhesion blockers (D-mannose, available in health-food stores and online, is occasionally used as preventive therapy)"
Comments:"UTIs caused by E. coli are initiated by adhesion of the bacteria to mannosylated receptors in the uroepithelium by means of FimH adhesin located on type 1 pili; theoretically, mannosides could block adhesion; however, D-mannose has not been evaluated in clinical trials"
Unless a review or better surfaces, dated since March 2012, I would use that as the basis of a statement.LeadSongDog come howl! 21:17, 2 April 2013 (UTC)
I appreciate everyone who has written positive (and negative) replies regarding inclusion of some kind of medical use of d-mannose on it's article in this or other way. However, this was given by me only as an example, I didn't intend this particular article to become a point of discussion.
I wanted to urge to take a broad look towards Alternative Medicine and address the issue that there are very few or few reliable secondary and even sometimes absence of even reliable primary sources for the field of the alternative medicine.
So my question is, are there any other sources that can be stated as reliable sources that can be applied to alternative medicine articles only?
My point is, the Wikipedia is here in order to serve the readers. The readers who would come to d-mannose article will probably very much interested in how it works for treating UTI's, is it really effective, possible side-effects, what kinds of d-mannose are on the market. Even if we will simply state that D-mannose is used by some people to treat UTI's that wouldn't be providing the information the readers are looking for, IMHO.
One editor has said: "Yes, that is allowed, if the wording is framed properly (making it clear it's an unproven claim) and the source isn't a commercial one." This statement let me ponder a bit and I have realized that in order for article about alternative medicine being informative, it's not relevant to wp:medrs only.
I'm not sure what policy has to discuss it, but something has to be done to remove this gap and allow the articles about alternative medicine to be informative, while not introducing any unproven medical claims. I would think that commercial sources should be allowed to be used, especially the review system such as that on Amazon. Some editors may scorn this idea, but I personally believe 200 reviews posted by independent users on Amazon much more than any secondary source of the best possible quality. Why? Because researches (that is not to discredit them) are made in controlled environments and may not always represent the practical world. The people who write reviews on Amazon are real people, who used the products themselves (unlike researches who mostly observe other people). What they write are true practical considerations.
In any case, even if you disagree with my opinion of using commercial sources too (I mean with attribution and proper wording showing that it's not a proven claim), something should be done to address the issue of AM on the Wikipedia. There is some point at which we shall strike the balance, and I believe the point should come out of consideration that the Wikipedia is here to provide the information the readers seek and it's upon us to develop such policies and considerations to make it possible. Ryanspir (talk) 15:55, 3 April 2013 (UTC)
There are a fair number of reliable secondary sources for alt med. If people wish to read amazon they are more than welcome to. But the consensus here is that amazon reviews are not reliable. Doc James (talk · contribs · email) (if I write on your page reply on mine) 17:21, 3 April 2013 (UTC)
Leaving aside your repeated assertion that several dozen academic review articles somehow counts as "very few", and ignoring the hundreds of books (most of which are also secondary sources) that cover alt med topics, what kind of information do you want to include that couldn't be supported from the secondary sources? "Some unknown customer on Amazon said it worked (or tasted good, or that he liked to take four capsules a day, or whatever)" is obviously not encyclopedic, so you must have something else in mind. WhatamIdoing (talk) 17:37, 3 April 2013 (UTC)
Noting above comments and given that it is not the job of Wikipedia to be a medical encyclopedia or directory for healthcare professionals of any kind, or a pharmacopeia, or a guide for self-medication, nor to do the job of public watchdogs or regulators such as FDA or MHRA, articles need to have some other informative value with suitable criteria for sources. Articles can usefully give information about such bodies and about medicine, medical education, and medical terms, including alternative medicine. So, to take Mannose as an example. Looking at the article, it is not evident why it is notable for a non-specialist other than for expanding the information elsewhere about metabolism. If mannose is being used as a basis for a product which, as a matter of fact, is available for a treatment, that is useful knowledge, provided it is mentioned what status it has for FDA (or similar) purposes while not seeming to guide medics or non-medics beyond that-- in the opinion of this layman, who would advise anyone to be wary of Amazon reviews... Qexigator (talk) 17:41, 3 April 2013 (UTC)
Like WhatamIdoing, I'm a little confused about what people actually want here, and moderately horrified (although not surprised) at the suggestion that we use aggregated Amazon customer reviews as the basis for our coverage of alternative medical products. MastCell Talk 18:09, 3 April 2013 (UTC)
I second what MastCell said! But I will say I am full out horrified at using Amazon reviews as a source on two levels levels; they are not a secondary source nor are they reliable, and just think about the SYN and cherry-picking that would be involved in using them! Any health content is subject to MEDRS and as always to WP:PSTS, be it about a drug or a dietary supplement.Jytdog (talk) 18:26, 3 April 2013 (UTC)

Trying to summarize what we have agreement on so far:

  • Referencing publicly-edited things like Amazon reviews is unacceptable. (This clearly fails Wikipedia policy, see WP:SELFPUBLISH prohibiting the use of "Internet forum postings" and the like.)
  • Articles can discuss the popular beliefs about or uses of things that have not been proven, given proper sourcing
  • Articles can and should discuss the fact that those uses have not been proven, and also any warnings that have been issued from regulatory bodies (FDA, etc.) regarding those unproven uses, given proper sourcing
  • WebMD should not be considered a reliable source, and we should update WP:MEDRS to reflect that
  • Can't see any consensus yet supporting any specific change to WP:MEDRS addressing alternative medicine in particular.

Zad68 13:25, 4 April 2013 (UTC)

Thanks Zad, great summary. Question - with respect to your third bullet, what do folks think about how to source a negative claim like "the fact that those uses have not been proven" if there is no secondary source that actually says that? Can we rely on common sense, or is that OR? And importantly, what if there is a source like this PMID 18950249 that says something actually IS useful even when there is no clinical evidence that it is? Jytdog (talk) 13:43, 4 April 2013 (UTC)
Doing WP:OR is, of course, a non-starter, if we're talking about doing things like running your own PubMed search, not finding any relevant results, and trying to put that in the article. Can't do that.

In the particular case of things like PMID 18950249, well I think from that point we just start doing our jobs of pulling sources, identifying the best ones, and summarizing them accurately in the article. Let's say we're talking about using PMID 18950249 to source content about the effectiveness of cranberries in preventing UTIs. The Alternative Medicine Review article says "Clinical research suggests the best natural options for long-term prevention include cranberry, mannose, and probiotics" and "Numerous clinical studies indicate several natural substances may provide effective prophylaxis in the case of recurrent infection. Nutrients and botanicals that have demonstrated the greatest effectiveness include cranberry, berberine, and probiotics." So the article might be used source "Cranberry juice may help prevent recurrent UTIs."

Let's look into it: The article is in a MEDLINE indexed journal, so OK, but the author is an "ND" ("Doctor of Naturopathic Medicine") and not an MD, and the journal itself is published by Thorne Research, which sells naturopathic products and is not primarily a scientific or academic publisher, so some flags start going up. On cranberries, the article was written in 2008, and makes reference to a a 2008 Cochrane review, which does indeed say "There is some evidence that cranberry juice may decrease the number of symptomatic UTIs over a 12 month period, particularly for women with recurrent UTIs." So the Alternative Medicine Review article isn't wrong, but if I came upon that in our Wikipedia article, and used it to find the Cochrane article, I'd probably leave the content alone and just switch the sourcing to the Cochrane review. Incidentally, that Cochrane review was updated in 2012 to say "Prior to the current update it appeared there was some evidence that cranberry juice may decrease the number of symptomatic UTIs over a 12 month period, particularly for women with recurrent UTIs. The addition of 14 further studies suggests that cranberry juice is less effective than previously indicated. Although some of small studies demonstrated a small benefit for women with recurrent UTIs, there were no statistically significant differences when the results of a much larger study were included." so I'd update the article to change the content to say "cranberry juice has not been found to be effective" and replace the 2008 Alternative Medicine Review article with the latest Cochrane review.

Basically, just keep doing what we do. Zad68 14:36, 4 April 2013 (UTC)

Thanks! But (a) you didn't address the hard question, which is, what if there is no secondary source that explicitly says that there is not evidence that X works for Y, and all you have is the absence of that.. and you do have sources saying that people use X for Y? That is the tough case. And (b) on PMID 18950249 - the question there arose from how it treats mannose -- it reviews extant studies, which are only in vitro and mouse studies, but in the last sentence the author recommends that people use mannose for long term UTI prevention... sorry i was not more specific. Jytdog (talk) 14:42, 4 April 2013 (UTC) (copyedited Jytdog (talk) 15:48, 4 April 2013 (UTC))
Jytdog, you're right, I didn't address hard questions, let's go: Regarding (a): My instinct on that is, if there's significant coverage on the topic in good secondary sources covering popular beliefs/usage, there's going to be a secondary source covering what the available evidence has to say about its effectiveness and safety. If there's no coverage of its effectiveness or safety, I'd bet the content about its popular use should probably be removed from the article per WP:FRINGE or WP:UNDUE. If I ran into it in an article and couldn't find good sourcing commenting on effectiveness and safety, I'd look into removing the article content; if the removal got challenged I'd discuss with the challenging editor; if I couldn't get consensus I'd probably notify WT:MED to get more eyes on it.

So let's move on to (b), regarding our specific example of mannose: PMID 18950249 lists only animal and in vitro evidence, and the review's summary is very qualified, "Numerous clinical studies indicate several natural substances may provide effective prophylaxis in the case of recurrent infection... Other interventions [i.e., ones without 'numerous clinical studies' supporting] with some positive clinical evidence but requiring further study include ... mannose." Per WP:MEDASSESS I wouldn't use this review's statement to source article content regarding the effectiveness or safety of mannose. So I did a little more searching and found PMID 22417256 (full text copy here), "Clinical practice: Uncomplicated urinary tract infection", a 2012 review article in the New England Journal of Medicine, which says of D-mannose: "available in health-food stores and online, is occasionally used as preventive therapy", and notes "D-mannose has not been evaluated in clinical trials". So, Wikipedia article content could be "D-mannose is used by some as a health food supplement intended to prevent recurrence of UTIs,(cite Alternative Medicine Review) but as of 2012, its effectiveness and safety for such use has not been studied in clinical trials.(cite NEJM)"

So I think we just have to take this on a case-by-case basis, dig deep for sources, and if sourcing regarding evidence isn't there, consider whether the mention is fringe or undue. I'm not sure we can write a blanket rule. Zad68 17:52, 4 April 2013 (UTC)

Adding: Even if we didn't find that NEJM article and only had the Alt Med Review article, based on the actual primary studies listed, the Wikipedia article might look like: "A 2008 review article in Alternative Medicine Review provided in vitro and animal study evidence in support of the possible effectiveness of mannose, and indicated that further study was needed." I think that would fairly represent it. Zad68 18:01, 4 April 2013 (UTC)
thanks!! that makes total sense. i have recently been looking at a lot of phytochemical articles and kept running into these kinds of problems. that NEJM article was a great find on d-mannose btw. :) Jytdog (talk) 18:29, 4 April 2013 (UTC)
I'm a little wary of making blanket pronouncements about borderline sources like WebMD. There are probably instances where WebMD can be a useful and valuable source. At the same time, it shouldn't be used to source extraordinary claims, nor should it be used to "rebut" the conclusions of higher-quality sources. For these kinds of sources, we need good editorial judgement to know when to use them and when not to. If we try to legislate the question one way or the other with regard to specific borderline sources, I worry that we're creating more problems than we solve. But that's just my 2 cents. MastCell Talk 17:15, 4 April 2013 (UTC)
  • Count me as not in agreement that WebMD is unreliable. This gets a little too far into the weeds of micromanagement. WebMD is one of those sources that can cover basic stuff that academic journals tend to skip over. Use with caution, sure, and maybe cite the NYTimes article in the guideline [3]. Research on alternative medicine is a complex topic, but you can take a look at the CDC project (in 2003, 208 treatment-condition pairs, 58% with at least one RCT) and the Cochrane review survey which I added to Alternative_medicine#Efficacy a few years back, which identifies a fair amount of research. Incidentally, Alternative Medicine Review ceased publishing in 2011 or so, and my experience is that its competitors (Scimago lists 55 CAM journals) are typically less neutral. II | (t - c) 17:24, 4 April 2013 (UTC)
  • I'm also unwilling to make blanket pronouncements about WebMD. It might be a lightweight source, but even lightweight sources can support lightweight claims. WhatamIdoing (talk) 16:48, 7 April 2013 (UTC)

I do *NOT* propose to consider Amazon reviews as reliable sources. But they do contain practical information that is of interest to the readers who are reading the alternative medicine articles. I think therefore that such articles should include secondary reliable sources, if available; primary reliable sources, if possible and practical information (such as a *summary* from Amazon reviews or information sourced to the reviews). If a certain Wiki editor will read 200 reviews for D-mannose and will be able to summarize them in the article, in the proper wording, I think he will be giving the readers the practical side of d-mannose use and have served the readers well. Ryanspir (talk) 10:11, 7 April 2013 (UTC)

And we avoid using non reliable sources. Doc James (talk · contribs · email) (if I write on your page reply on mine) 16:40, 7 April 2013 (UTC)
Why should we be including "practical information"? Wikipedia is WP:NOT#HOWTO. WhatamIdoing (talk) 16:48, 7 April 2013 (UTC)
Ryanspir, you seem to be contradicting yourself. If Amazon reviews are not reliable sources (as per your first sentence), how can any content be sourced to them (as per your 2nd and 3rd sentences)? (real question, not rhetorical) Jytdog (talk) 20:25, 7 April 2013 (UTC)
The answer is here: "Alternative medicine is exactly within the ambit of WP:MEDRS, and in fact is one of the major reasons the guideline was created. As my learned colleague Spinningspark rightly states, you can absolutely address the claims that a specific source makes if you identify them as claims made by that source, and such viewpoints are adequately balanced with the mainstream perspective (which generally will meet MEDRS). Hope this helps." — by editor Mendaliv.Ryanspir (talk) 08:24, 22 April 2013 (UTC)
Ryan you quoted someone else and did not attribute the quote properly, you also did not sign and so you made it look like what you posted came from someone else. Please take more care to attribute your use of the the words of others more carefully. Zad68 14:27, 17 April 2013 (UTC)
Thanks. Hope it is better now. Ryanspir (talk) 08:24, 22 April 2013 (UTC)

Alt med reliable sources

Assuming that a section about alternative medicine will be adopted or certain alt med specific guidelines will be created, are there are any reliable sources that can be used for alt med in your opinion that are not currently included in MEDRS?

For example Alt Med specific sources that doesn't apply to Medicine articles, but not necessary.

I'm well aware that some editors do not approve to give any special considerations to alt med and determinations to threat it as any biomedical info. Please see this section as not a discussion about it. That's why I have said: "Assuming" and assuming only. Only positive propositions for consideration please. Ryanspir (talk) 15:20, 11 April 2013 (UTC)

I don't understand your second paragraph. What isn't a medicine article? What isn't necessary? Please give an example. WhatamIdoing (talk) 15:46, 11 April 2013 (UTC)
I mean are there some new possible sources that could be applied both to medicine and alternative medicine; or are there any source that theoretically may be considered reliable sources for alt med (while not reliable for normal medicine articles). Ryanspir (talk) 16:57, 12 April 2013 (UTC)
No. This guideline is about appropriate sources for health-related content. Whether the content addresses evidence-based medicine or alternative medicine, or aspects of biology or psychology with health implications, the standard is the same: high quality independent scholarly secondary or tertiary sources (and very, very, very rarely primary sources). --Anthonyhcole (talk · contribs · email) 17:08, 12 April 2013 (UTC)
Yes, I got impression that this is the case. Therefore I have proposed for removal of two references: quackwatch and LA times article in the article Medical uses of silver that do not have a corresponding reliable secondary source that proves that cs is not effective for treating the mentioned conditions. Ryanspir (talk) 13:00, 13 April 2013 (UTC)
I disagree with your application here (i.e. let's not apply this out of context): for notable events the LA Times and other (non-medical) reliable sources could be a reliable source for such events that do not explicitly or implicitly make biomedical claims; also, Quackwatch has already been vetted specifically as a reliable source on biomedical topics, hasn't it? -- Scray (talk) 17:12, 13 April 2013 (UTC)
Yes, Quackwatch has long had consensus as reliable for discussion of alt med topics. Quackwatch and the LA Times article are both secondary sources and are perfectly reliable for the material they are used to support. This appears to be another step in a long line of steps of this WP:SPA to promote alt med applications of silver in this article. Yobol (talk) 17:47, 13 April 2013 (UTC)
Could you please provide a link to where Quackwatch was vetted specifically as a reliable source for alt med topics? So far I have only seen a decision of Arbcom in which it was denied to appear in an article.
Regarding LA Times can you please provide where it was decided that its a secondary source? According to WP:MEDRS it's a primary source. "For Wikipedia's purposes, articles in the popular press are generally considered independent, primary sources." Thank you. Ryanspir (talk) 14:42, 15 April 2013 (UTC)
What scray said was "for notable events the LA Times and other (non-medical) reliable sources could be a reliable source for such events", so he is not saying is a secondary source, but that it is a reliable source for non-medical content, for which they are tipically reliable. You should remember that not every fact in a medical article is medical, so for example for a social event related to the article is a reliable source. Nevertheless source deciding is not black and white and in many cases should be decided in a case by case basis, so I would not say that quackwatch or NYT are or are not reliable, but that it depends.--Garrondo (talk) 15:06, 15 April 2013 (UTC)
A lot of editors have trouble keeping secondary and independent straight. That's why WP:Secondary does not mean independent exists. It's also worth having a look at WP:USINGPRIMARY, although that's primarily focused on non-medical sources.
However, these sources, as used in Medical uses of silver, are authoritative: when the statement in question is "Source X said ____", you will never find a source more reliable than Source X itself to support it. The specific claim for the LA Times, by the way, is legal, not medical: "a fraud with a long history". WhatamIdoing (talk) 20:47, 15 April 2013 (UTC)
Allow me respectfully to disagree. Currently it says: "The Los Angeles Times stated that "colloidal silver as a cure-all is a fraud with a long history, with quacks claiming it could cure cancer, AIDS, tuberculosis, diabetes and numerous other diseases." That implies that cs is ineffective because they say: "quacks claiming it could cure". That is a biomedical determination for which an adjunct reliable secondary source must be cited per WP:MEDRS in my humble understanding. Ryanspir (talk) 13:50, 17 April 2013 (UTC)
You may of course disagree, but you are absolutely alone in your view here. Zad68 14:29, 17 April 2013 (UTC)
That quotation is technically a claim that the LA Times said this, not a claim that the LA Times is correct. WhatamIdoing (talk) 15:54, 18 April 2013 (UTC)
Yes WhatamIdoing, you are right. LA article provides biomedical information by saying so. Ryanspir (talk) 10:05, 22 April 2013 (UTC)
Ryan you have an amazing ability to "agree" with things people have not said. Zad68 13:56, 22 April 2013 (UTC)

New "open access" journals may jeopardize medical information system.

See related discussion.

From "Consumer Health Digest," April 18, 2013.

The New York Times has reported problems associated with "open access" journals that operate unethically and have low editorial standards. [Kolata G. Scientific articles accepted (Personal checks, too). The New York Times, April 7, 2013] Open-access journals publish the full text of articles on their sites, where they can be accessed free of charge. Researchers have expressed alarm that some journals of this type will print seemingly anything for a fee. The article also noted:

  • Non experts—and even experts—doing online research will have trouble distinguishing credible research from junk.
  • Jeffrey Beall, a research librarian at the University of Colorado in Denver, has posted a list of "potential, possible, or probable predatory scholarly open-access publishers." There were 20 publishers on his list in 2010. Now there are more than 300. He estimates that there are as many as 4,000 predatory journals today, at least 25% of the total number of open-access journals.
  • Academics who become associated with such journals without realizing their true nature risk tarnishing their reputation.

END QUOTE

We need to be thinking about this type of problem when we cite journals. That's an awful lot of dubious journals! We may end up having to blacklist some of them. -- Brangifer (talk) 02:45, 21 April 2013 (UTC)

Also being discussed here, among other places. MastCell Talk 18:46, 22 April 2013 (UTC)
Good! -- Brangifer (talk) 02:23, 23 April 2013 (UTC)
And this at a time when the Nature journals are upping their standards. http://www.nature.com/ni/journal/v14/n5/full/ni.2603.html. Part of what they are reacting to, is 2 recent studies from scientists in industry who tried to replicate important findings published in primary studies by basic scientists (to see if their companies should invest serious $ in developing drug discovery projects based on them), and found that they could only replicate about 20% of them! One is from Amgen (http://www.nature.com/nature/journal/v483/n7391/full/483531a.html#affil-auth) and another is from Bayer (http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html). All the more reason that no health related claims should be backed up by primary sources only. And yes, we should be on the lookout for content sourced from the journals discussed above. Jytdog (talk) 19:01, 22 April 2013 (UTC)

See related discussion.

Review of Monsanto's Roundup herbicide

Hi there, I'm hoping this is the right place to ask for an opinion. My work at the Monsanto page was reverted here with the edit summary: this source fails MEDRS - health-related content cannot be based on it. Also there are much better and more comprehensive reviews on Glyphosate toxicity, if this were the place for it. which it is not.

The reason I placed this information at the Monsanto article: the secondary source I referenced was entitled Heavy use of herbicide Roundup linked to health dangers. Monsanto makes Roundup; the Monsanto article covers this product, and to my knowledge there is no other article where Roundup is covered. In my opinion, this information belongs at the Monsanto page as well as the more specific article about the chemical in Roundup, glyphosate.

The sources used were: a Reuters, and a primary study from Entropy.

Do these sources meet MEDRS, or is this possibly "a public-interest issue, not strictly a MEDRS one" as SlimVirgin suggested when I sought advice? Thank you for your time, petrarchan47tc 00:23, 30 April 2013 (UTC)

First, medical claims should be sourced to medical writing, not newspaper accounts of them. Second, we should not report primary but use secondary sources that explain the significance they have. It is a MEDRS issue, because it implies there is a medical issue, "Roundup linked to health dangers". TFD (talk) 00:52, 30 April 2013 (UTC)
We avoid individual studies because it's so easy to pick out only the studies that happen to agree with our personal opinions.
You need a secondary source, which for toxicology usually means a review or a reference work, so go to http://www.pubmed.gov and type glyphosate toxicity into the search box. On the left side of the screen, click 'reviews' under "Article types". You'll find about ten that were published during the last ten years. PMID 22683395 says there's no particular relationship to cancer, and PMID 21798302 says that any relationship to non-cancer problems like Parkinson's is weak at best. If you're interested in food contamination, then PMID 21541850 looks interesting, but there are no links to the full paper. WhatamIdoing (talk) 01:51, 30 April 2013 (UTC)
I was the one who reverted the addition of that information. Here is what I have written about the Entropy article: "The key idea of MEDRS is that health-related content needs to be based on very secure foundations, the best we have, that expresses the consensus of the medical community. In this case, the journal is not a biomedical journal or textbook. The authors are not part of the medical community. What they present in this article, is not the medical consensus, nor even primary biomedical experimental research, but instead they present novel hypotheses they have generated based on their review of primary and secondary biomedical literature. This is not a secondary source --a review -- in the standard way we think about them -- it is really a primary source, presenting what I would call "theoretical biomedical research" (there are actually a few journals now for theoretical biology). But you can see here that on all three levels -- journal, authors, and content, the source fails MEDRS. This is not a reliable source, presenting the consensus of the medical community, so no content based on the Entropy article has any place in Wikpedia. " And I really believe this is true -- this journal and this article is not a valid source under MEDRS for health related content. Same goes for news reports about this article. Jytdog (talk) 02:11, 30 April 2013 (UTC)
I am a bit wary though of the use of MEDRS here, certainly the paper can by no means be used to claim anything as established or even as a medical consensus, but that doesn't necessarily it cannot be used at all as it depends how you phrase and attribute its claims in WP. The actual phrase in policy is : "Speculative proposals and early-stage research should not be cited in ways that suggest wide acceptance. " (see also the paragraph it belongs to).--Kmhkmh (talk) 06:27, 30 April 2013 (UTC)
If you have sources for your claims about the journal, then Entropy (journal) definitely needs to be expanded. WhatamIdoing (talk) 03:37, 30 April 2013 (UTC)
Not sure what you mean! It is a journal of "entropy and information studies" - not a biomedical journal. Jytdog (talk) 03:45, 30 April 2013 (UTC)
btw, here is what one of the authors says about Entropy and the work she has published there. Sounds pretty fringey to me: "Note: Entropy is an Open Access journal that is willing to publish novel hypotheses regarding biochemical and biophysical phenomena, which can help the community break out of its current straitjacketed research paradigm. The papers below, many of which were published in Entropy's Special Issue on Biosemiotic Entropy: Disorder, Disease, and Mortality, cover several topics relating environmental toxins to disease, as well as the revolutionary concept that endothelial nitric oxide synthase (eNOS) synthesizes sulfate as well as nitric oxide. The papers were subjected to rigorous review by experts who were not beholden to industry influence. These papers collectively explain how widespread cholesterol sulfate deficiency throughout the body is behind most modern diseases and conditions." from here: http://people.csail.mit.edu/seneff/ which I got from here: http://ksj.mit.edu/tracker/2013/04/discover-blogger-keith-kloor-stumbles-ne This just points up what I am saying, that this article is far far from the medical consensus. And please note that this is an open access, fee for publishing journal like those discussed above. To be fair, it is not currently on the list referenced above. Jytdog (talk) 04:39, 30 April 2013 (UTC)

There was already a preliminary discussion about the reliability of the source and the red flags it raises at the Monsanto article (see Talk:Monsanto#entropy_study/bad_science?. Personally I think the paper is at first glance "formally reliable" reliable as it is was published in a peer reviewed academic/scientific journal. But is raises various red flags, such as its weeping claims without any new experimental evidence and the fact that is was published outside the main domain it primarily belongs too, that is a journal in the field of chemistry, biology, medicine or agrarian and environmental sciences. The same issue with the who is an established scientists but in computer science rather than in those domains.

However aside from personally assessing the paper's potential problems by WP editors, WP should if possible mainly rely on external source if available. Unfortunately I haven't seen scientific review of the paper yet, however there are a view journalistic reviews available that could be consulted. I copied the material from the Monsanto discussion, feel free to extend it, so that everybody discussing here has on overview about what external sources say about the paper:

--Kmhkmh (talk) 05:13, 30 April 2013 (UTC)

  • Derek Lowe (chemist): Is Glyphosate Poisoning Everyone?. Corante, April 30, 2013 "After spending some time reading this paper over, and looking through the literature, I've come to a conclusion: it is, unfortunately, a load of crap." Mmangan333 (talk) 14:40, 3 May 2013 (UTC)
P.S.: On a site note, while I see this paper rather skeptical, it might be worth to look at the articles it cites. Though there are probably not so many well researched scientific publications aboutissues of Glyphosate, one has to keep in mind that this is highly politicized field with (almost) all research funded by parties with a high financial stakes in Glyphosate, i.e. there is one hand a lack of really neutral high quality research, but on the other hand there is still a lot of journalistic/"anectdotical" evidence (local health issues popping up in areas with large scale glyphosate use in the Latin America and Asia in particular). This evidence is nowhere near a scientific result/knowledge, but it is well known enough to warrant mentioning in the concerned articles on glyphosate or monsanto.--Kmhkmh (talk) 05:27, 30 April 2013 (UTC)
A note about the articles above: I am shocked that a discussion about MEDRS is pointing us to two bloggers - Tamar HaspeL for the Huffington Post, and "Discover blogger Keith Kloor". I don't understand how bloggers could have any place in this conversation. petrarchan47tc 05:49, 30 April 2013 (UTC)
As I said already there are only some journalistic reviews available yet. "Blog" is describing the technical format/software, which has little meaning. To formally judge the merit of a "blog", you would need to assess the reputation of the authors and whether the "blog" has a particular reputation or is associated with publisher with a certain reputation."Blogs/Bloggers" as unusable/unreliable for WP refers primarily to private blogs by private citizens without a particular reputation (that is your average blog on wordpress & co). This is similar to websites, (average) private websites are normally unreliable, but that doesn't mean any website as such is unreliable. Note strictly speaking any online content is a "website" of some sort. Keeping that in mind both "bloggers" are associated with regular publications (Huffpo and Discover magazine) and can probably considered to be journalists and their publication as a journalistic review of the Samsel/Seneff paper. Not more not less, i.e. they provide some input/inside but certainly no authoritative opinion on the paper, a bit more authoritative assessment however might come from the knights journalism at MIT publication which however essentially criticizes the same points. Also note, that at the current stage the only intended usage of all of them is to help assessing the Samsel/Seneff paper, we are not discussing about sourcing anything with them.--Kmhkmh (talk) 06:20, 30 April 2013 (UTC)
But these particular bloggers' reputations have not been established here, yet we are referring to them for opinion on sources. Baffling. petrarchan47tc 18:47, 30 April 2013 (UTC)
Petrarchan, it's actually much simpler than that, the journal Entropy is not MEDLINE indexed, see here. If the journal is not even MEDLINE indexed it fails to meet even the minimum entry requirements for a WP:MEDRS-compliant reliable source for biomedical information, and we do not even have to consider what bloggers might have to say about it. Beyond that we have multiple recent review articles in MEDLINE-indexed journals covering the topic of the safety of glyphosate in humans, see the several PMIDs brought up by myself and Whatamidoing elsewhere in this section, there is no need to even consider looking at Entropy for this information. Zad68 19:05, 30 April 2013 (UTC)
Bloggers shouldn't have been a part of any serious conversation, unless they are shown to meet WP:RS. (I have to admit, I'm fascinated to hear that wide-use of a herbicide is completely safe for humans.) I must admit, it is confusing that no reputable source has taken issue with the study, and that it fell on the shoulders of bloggers and Monsanto to write up refutations. I continue to be baffled. Thanks for your time, though. petrarchan47tc 19:34, 30 April 2013 (UTC)
Again, you confuse the use of sources with the assessment of sources. As far as glyphosate is concerned I don't recall anybody here claiming it to be safe, but just that the sweeping speculation of the entropy paper have no place in WP.--Kmhkmh (talk) 02:36, 1 May 2013 (UTC)The discussion here is not about merits or dangers of glyphosate but about the merits of the entropy paper and whether it is an acceptable source for WP or not.--Kmhkmh (talk) 02:36, 1 May 2013 (UTC)
Oh, you are right, my apologies. I misread Zad's comment. petrarchan47tc 20:17, 1 May 2013 (UTC)
You're welcome, hope you found this conversation useful. Whether it's really true or not that glyphosate is completely safe for humans isn't actually something Wikipedia worries about; we only repeat what reliable sources say. I would guess that the reason no really reputable independent source has taken up issue with the Entropy article is that the article carries so little authority that nobody cares enough about it to reply. The article really doesn't carry much more weight than a self-published eBook. Zad68 19:47, 30 April 2013 (UTC)
In the category of chemicals used for weed control and their safety in humans, I understand that they're using dihydrogen monoxide to kill weeds these days. The suppliers keep saying it's safe, even though that's been linked strongly to fatal outcomes, especially in young children. WhatamIdoing (talk) 20:07, 30 April 2013 (UTC)
What's even crazier is that homeopaths have found that it's even more potent in microscopically tiny doses... if you give weeds just a few molecules of it (and allow them no more) they'll die as well. It's obviously potent stuff! And I'd love to laugh at "The suppliers keep saying it's safe..." but I've started hanging out at Talk:Water fluoridation so now my brow only furrows with worry... Zad68 20:27, 30 April 2013 (UTC)
SlimVirgin's advice that "this is a public-interest issue, not strictly a MEDRS one, so I would allow it, though I might write it differently." is dreadful. The writing and the source break our core policies, never mind the specifics that MEDRS might add. As others have said, this paper is original-thought published in a journal with zero reputation for the field of toxicity and where one of the authors appears to be a computer scientist. Fails WP:V straight away. It seems to regard RoundUp as potentially liable for all modern human ills. How many WP:REDFLAGs are needed? Per WP:WEIGHT is this appropriate to the article Monsanto? Or even the glyphosate article? As a primary source, we need secondary sources to give this viewpoint any weight. The Reuter's article is not such a secondary source. Reuters is a agency that (among other activities) uncritically packages up press-releases into "articles" that appear to be proper journalism and can be copy/pasted into newspapers for a fee. Such articles tell us nothing about what experts in the field have to say, or about what influential thinkers have to say, or the public thinks.... The issue of whether Monsanto's products cause some of our modern ills is a "public-interest" one, but it takes more than a regurgitated press-release to establish what commentators (scientist, environmentalist, politician or journalist) have to say on the issue. Colin°Talk 12:44, 30 April 2013 (UTC)
Though I agree on your assessment of this particular source and the red flags, I disagree somewhat on Slim Virgin's advice being dreadful, because imho she has a point there. That is we may need to cover material for which no peer reviewed medical resources are available yet, but for which other "reliable" sources and media coverage does exist. We can of course not give claims in such sources a "factual" status, that they don't have, but we can't completely ignore them either In particular in the case of Round-up/Glyphosate isuues there is independent (not reurgitated) "quality" journalism available, the Reuters article is just not a part of that. Such independet journalistic material is for instance a recent AJE documentary "Bad Seeds" or the french Documentary "World according to Monsanto".--Kmhkmh (talk) 13:09, 30 April 2013 (UTC)
Re-read what she said. The reason she would allow the reverted text was that she felt this issue wasn't covered by MEDRS. Totally ignoring all the other policy pages which make it quite clear this source and the text drawn from it should not appear on the Monsanto article. Now there may well be solid reliable non-medical-journal/textbook sources that discuss the Roundup Makes People Unhealthy issue from various angles but neither the source(s) given nor the text drawn from them is acceptable. petrarchan47tc asked SlimVirgin for her take on the "sources and content" with the angle that the "same excuse" (i.e., MEDRS) was being used to remove this. petrarchan47tc did not ask SlimVirgin if she was aware of better sources or for her view on the subject itself. So SlimVirgin basically agreed that because she felt there was a "public-interest" angle to the issue, all other critical faculties necessary for judging ones sources and facts become irrelevant. That's "Daily Mail" writing and doesn't belong here. Colin°Talk 13:30, 30 April 2013 (UTC)
We are talking besides each other it seems. I don't support SlimVirgin's argument applied on this particular source, but I do support that line of argument in general. That doesn't mean the general criteria on sources can be lifted but just the specifications/extensions by MEDRS may be lifted in a particular context due to the subject in question having a "public interest" angle or more generally a non purely medical aspect to it.--Kmhkmh (talk) 13:44, 30 April 2013 (UTC)
"I do support that line of argument in general" - what line of argument is that? Because SlimVirgin didn't advocate any argument beyond her claim that MEDRS didn't apply. It does apply in that a health/medical claim is being made (X "could cause" disease Y). Once we step outside the medical aspect, we can consider this speculation in the same way as we consider any original thought on any subject. Is it notable? Is it mentioned by sources writing about the article topic? Are our sources reliable? Etc. I think you are reading the "public-interest issues might draw on a variety of sources" aspect into what SV wrote. She didn't say that at all. Colin°Talk 14:37, 30 April 2013 (UTC)
Kmhkmh, Colin is right. MEDRS is clear that it applies only to health content in a given article. And I don't see how you can say there is any non-health related content that could be sourced from the Entropy article. With regard to any supposed "public interest exception" to MEDRS - there is none, and this is the opposite of what should hold. The more public interest there is in a health related issue the more important it is that MEDRS apply. So... I don't get your point. Jytdog (talk) 13:57, 30 April 2013 (UTC)
I'm not arguing to source anything with the entropy article, due to the red flags it should be used in WP at all. Nor should the Reuters article be used to spource anything. However other "quality Journalism" pieces may be usuable and so might be some of the sources cited in the entropy article. In short if somebody wants to source suspected/potential medical issues og glyphosate in the glyphosate article he (and probably should) do that, but without using the Samsell/Seneff paper or the Reuters article. You might read that as an advice to Petrarchan47. There are other acceptable sources out there describing various health issue with Glyphosate (with less sweeping claims though), so there is no need for him to rely on this particular questionable paper.--Kmhkmh (talk) 14:12, 30 April 2013 (UTC)
We have 20 recent review articles on the stuff. Why the need for primary sources? Doc James (talk · contribs · email) (if I write on your page reply on mine) 14:03, 30 April 2013 (UTC)
Reviews on what exactly? And could you point out those reviews explicitly please.--Kmhkmh (talk) 14:14, 30 April 2013 (UTC)
jmh, the "why" is that the Entropy article is recent and got some buzz in environmental circles, so several editors who are not usually involved in the Monsanto and glyphosate articles wanted to add content based on it, to those two articles. I've been trying to make clear that this source fails MEDRS and that is how the conversation got started here, when one of the editors I reverted brought it here for discussion. You are right that there is no need at all for this source. It is just being driven by WP:RECENTISM Jytdog (talk) 14:17, 30 April 2013 (UTC)
Kmhkmh, if you look above, whatamidoing gave instructions for finding recent reviews of glyphosate toxicity. Jytdog (talk) 14:17, 30 April 2013 (UTC)
Glyphosate targets a metabaolic pathway that exists only in plants and not in animals so there would be no initial expectation that it would have a harmful effect on humans. Not saying these are all the best quality (haven't really checked) but for starters take a look at:
  • PMID 22683395 "no consistent pattern of positive associations indicating a causal relationship between total cancer (in adults or children) or any site-specific cancer and exposure to glyphosate"
  • PMID 22202229 "the available literature shows no solid evidence linking glyphosate exposure to adverse developmental or reproductive effects at environmentally realistic exposure concentrations"
  • PMID 21798302 "no evidence of a consistent pattern of positive associations indicating a causal relationship between any disease and exposure to glyphosate"
Per WP:EXTRAORDINARY you'd have to come up with extremely solid sourcing to have the article state there's been found to be a health risk to humans, and the Entropy article and popular-press items do not rate. Maybe SlimVirgin meant "although there's no scientifc consensus of a health risk to humans, there's a popular perception that there is, and this article might be used to describe that" but I don't want to put words in SV's mouth. Zad68 14:36, 30 April 2013 (UTC)
I think it is probably time to separate the discussion of those 2 particular sources (Entropy & Reuters articles) from glyphosate related issues and other sources for it. The latter should be discussed on the glyphosate talk page and if another controvery about a particular source arises, we can discuss that here. But we shouldn't the general glyphosate discussion here.--Kmhkmh (talk) 14:42, 30 April 2013 (UTC)

Sorry if I'm missing it, but is there anything left to discuss here? I think there's a solid consensus that Jytdog's revert per WP:MEDRS was correct. I don't think anybody is arguing WP:MEDRS applies to sources such as popular press for non-biomedical information about general discussion of a subject, such as the possibly incorrect popular perception of something being unsafe when medical consensus is that it is safe. Anything I'm missing? Zad68 16:30, 30 April 2013 (UTC)

Seems to me that we are done, too. There are some off-topic discussions running but the main question of whether the article in Entropy passes MEDRS is resolved I believe. Thanks for summarizing, zad. Jytdog (talk) 18:51, 30 April 2013 (UTC)
I believe this information should be in the Monsanto article. It could look something like how the Washington Post covered it yesterday:
Roundup is tied to infertility and cancer; herbicide’s maker calls it safe
Heavy use of the world’s most popular herbicide, Roundup, may be linked to a range of health problems and diseases, including Parkinson’s, infertility and cancers, according to a new study.
The report, published this month in the online journal Entropy, said evidence indicates that residues of glyphosate, the chief ingredient in Roundup and other weedkillers, has been found in food.
Monsanto and other leading industry experts have said for years that glyphosate is proven safe and has a less damaging impact on the environment than other commonly used chemicals. petrarchan47tc 19:34, 30 April 2013 (UTC)
I agree that these sources don't meet WEDRS. (But feel it should be mentioned, perhaps in a "Monsanto controversy" section.) petrarchan47tc 19:37, 30 April 2013 (UTC)
I don't see how any content based on the Entropy article can come into wikipedia. Jytdog (talk) 19:49, 30 April 2013 (UTC)
You could only justify mentioning the Entropy article in a controversies section if you had an independent reliable secondary source that provided a general overview of Monsanto controversies, and it highlighted that Entropy article as a noteworthy enough topic within Monsanto controversies to justify including it. Per the above discussion I do not get the impression it would be noteworthy enough to include. You can make a list of many, many things that can be said regarding controversies concerning Monsanto, but I don't think that particular article is in that list's top 10 items. Zad68 19:55, 30 April 2013 (UTC)
Essentially any editor that stated an opinion here to answer your question has declined the use of the Entropy article as an acceptable source, so that issue seems settled and the discussion can be closed here. The Washington Post entry btw. is just a copy that equally unable Reuters piece. If want to point out health issues of Glyphosate there are other sources you could (and would need to) resort to and the article on glyphosate covers quite a variety of them already anyhow. However an extended discussion on that belongs on the talk page of glyphosate and not here.--Kmhkmh (talk) 20:42, 30 April 2013 (UTC)

The Entropy article is peer-reviewed and thus sufficient to support its conclusions being present in the article about Monsanto. I was shocked to find that there was absolutely no mention of genetically modified organism (GMO) in the article; an amazing lapse. Professor Daniel W. Schneider of the University of Wisconsin at Madison ties together the parts of this issue, telling his readers in Hybrid Nature: Sewage Treatment and the Contradictions of the Industrial Ecosystem (2011, MIT Press) that Monsanto invented the business strategy of selling both glyphosate pesticides (Roundup) and glyphosate-resistant GMO crops, the strategy being immensely successful. Schneider discusses how the GMO crops may be harmful to humans and the environment because of how much glyphosate is poured on them during the growing season. The Entropy study is not the only indicator of the total problem. As well, the Schneider book ties the problem firmly to Monsanto, which means we should be not to be ghetto-izing the controversy at the glyphosate article—the issue should absolutely be raised at the Monsanto page. Binksternet (talk) 00:23, 1 May 2013 (UTC)

Binkster this is the Talk page for the MEDRS guideline. Not about Monsanto or Glyphosate, and the question that was put, is "is the Entropy article acceptable under MEDRS". Nothing you wrote is relevant to MEDRS. Please read it and the discussion above. And you cannot use a book published in 2011 to back up an article published in 2013. The Entropy article is not acceptable for health content in wikipedia. Please let it go. Jytdog (talk) 01:29, 1 May 2013 (UTC)
What's amazing to me is the number of people who see the words "peer-reviewed" and somehow believe this is a magic incantation making that source reliable. There are a number of red flags with this particular source, and it should not be used for any medical information on Wikipedia. Yobol (talk) 01:34, 1 May 2013 (UTC)
(e/c) Binks, sorry, it simply isn't true that just because something is "peer-reviewed", that by itself makes it qualified for inclusion. The WP:MEDRS guideline basically says for journal articles, being "peer reviewed" is a necessary but not sufficient condition.

Regarding "no mention of GMOs", I left a message for you at SV's page - the article covers specific subtypes of GMOs like GM seeds and GM crops, with appropriate Wikilinks. The article's coverage of the GM topic may not be ideal but it's incorrect to say there's a complete "lapse" of such coverage in the article. (Disclosure: I have never edited that article or its Talk page and really haven't had any interest in doing so.) This and the Schneider source you bring should probably be discussed at Talk:Monsanto and not here at WT:MEDRS. Zad68 01:38, 1 May 2013 (UTC)

Except MEDRS actually says no such thing. Peer reviewed is neither necessary nor sufficient per se. It is however the highest for "formal" requirement we have, but it nevertheless may not always be sufficient. The entropy paper is such a case, it is peer reviewed, but due its lack of reputation of the publisher & peer review process as far as domain knowledge (biomedical, biochemistry) is concerned it is not sufficient.--Kmhkmh (talk) 02:50, 1 May 2013 (UTC)
Kmh, we're in agreement on the Entropy paper. It's peer-reviewed but the other factors concerning it make us conclude it is not sufficiently reliable to source biomedical information. If it were not even peer-reviewed, the conversation regarding its fitness would have been much shorter, right? ("It's not even peer-reviewed, not usable as proposed per WP:SELFPUBLISHed.")

WP:MEDRS does mention the value of peer review many, many times, and includes: "Every rigorous scientific journal is peer reviewed. Be careful of material published in a journal that lacks peer review or that reports material in a different field." In practical terms, a non-peer reviewed journal article is a non-starter to source biomedical information. I guess it's possible for something we would consider to be a reliable, authoritative source for biomedical information (on par with the top of WP:MEDASSESS) could be published in a journal without a peer review, but that would be exceptional and for my own education I'd like to see such a case... and even if there were such a case, wouldn't there be peer-reviewed articles we could use instead? WP:V, the underlying policy, directs us to look for "a reputation for fact-checking" and in the academic world, and in the biomedical sciences in particular, that means peer review. Zad68 03:21, 1 May 2013 (UTC)

There is a no disagreement about the importance of peer reviewed for MEDRS (or RS), but you were talking about necessary and sufficient conditions (if I understood you correctly), in particular claiming peer reviewed being a necessary condition and MEDRS simply doesn't state that (see Necessity and sufficiency).--Kmhkmh (talk) 03:40, 1 May 2013 (UTC)
Yup you understood me correctly, and it is true WP:MEDRS does not state it explicitly, but in practice, that's how we edit when we refer to WP:MEDRS. I qualified my statement as "The WP:MEDRS guideline basically says..." and by "basically says" I mean "that's the bottom line of how we interpret it". This is the same sort of thing regarding primary sources: technically WP:MEDRS doesn't forbid them, but in the practical reality of day-to-day editing, that's how we apply it (with some rare exceptions). I'm a very practical person, WP:MEDRS says what it says, but all that really matters is what kinds of edits we end up making or supporting or not supporting when we refer to it. Zad68 03:49, 1 May 2013 (UTC)
MEDRS doesn't require peer review, and it shouldn't. The requirements depend on the statement that you're making. If the statement is moderate in its scope, like "antibiotics do not kill viruses" or "the most common cancers in children are leukemias and brain tumors", then we'd accept a plain old magazine article or even a page from a charity website. The source needs to be strong enough to support the claim. A moderate claim needs only a moderate source (although you are free to use a stronger one). An extraordinary claim (e.g., that one chemical causes a dozen unrelated diseases) requires an extraordinary source.
Also, the opposite of peer-reviewed is not "self-published". Newspaper articles are not peer-reviewed, but they are not self-published, either. WhatamIdoing (talk) 16:32, 1 May 2013 (UTC)
WAID, I understand what you're saying here. My comments were targeted to the context of the conversation we'd been having: citing a journal article to source non-obvious biomedical claims (or at least biomedical claims that have been challenged, as they have been here). Although honestly I would not support sourcing "the most common cancers in children are leukemias and brain tumors" to a cancer charity website. If the charity sourced that information to a reliable source, OK, but then I'd use that source instead.

We were talking about journals... I understand newspapers are not "peer-reviewed" but they have fact-checkers who (are supposed to) verify the information published. Both types of publishing have their own models for doing the same thing. Anyway, WP:MEDRS advises not to use newspapers to source biomedical information of the kind being discussed in this thread - I'm pretty sure you'd agree with that, yes?

In the case of the particular journal article we've been talking about here: Entropy is an open access (pay for publish) journal. It is peer-reviewed, but if it were not, the only thing that would have stopped the glyphosate article from getting published would have been the discretion of the editorial board taking the money from the article's authors, recording studio owner Samsel and computer scientist Seneff, and with few repercussions for the journal's reputation if the article weren't all that great (Entropy is not MEDLINE indexed, and doesn't even advertise itself as publishing in the biomedical domain). Maybe my "WP:SELFPUBLISH" comment was a bit sharp, but I think my point is on target. Zad68 19:04, 1 May 2013 (UTC)

Like I said, you are free to use a stronger source than is required. (You must actually read that stronger source; it's never okay to cite the website's source when you only read the website.) But using a stronger source than necessary is never required.
Source selection is a means to an end. We aren't here to create impressive-sounding bibliographies. We're here to write a reasonably accurate and reasonably comprehensive encyclopedia. Source selection helps us with the "reasonably accurate" part. If a paper-only medical text, a free online magazine article, and an expensive journal article all give the same basic facts, then you are free to cite whichever one(s) you want. We only require the strongest sources for heavy claims. WhatamIdoing (talk) 21:19, 1 May 2013 (UTC)
I did say "If the charity sourced that information to a reliable source", and "using a stronger source than necessary is never required" is of course tautologically true... But I think we're starting to talk past each other a bit and I don't want us to get lost in the weeds of this side conversation here. We're both in agreement that the Entropy article was an insufficient source for the article content it was proposed to support, and we both offered articles like PMID 22683395 and PMID 21798302 to cover the topic of the safety of glyphosate instead. I think that's the important thing regarding the original question. Zad68 02:09, 2 May 2013 (UTC)
Our sources are never required to list their sources. It's in the FAQ. WhatamIdoing (talk) 20:27, 6 May 2013 (UTC)

There are ongoing efforts at the Monsanto article and the Glyphosate article to include content based on the Entropy article by new people who pop in and are not minding the ongoing discussion. Maybe it will die out soon but I would very much appreciate folks here watching those pages and helping deal with it so I don't run afoul of 3RR. Thanks! (The tox section of Glyphosate could also use some MEDRS love if anybody is interested.) Jytdog (talk) 13:08, 1 May 2013 (UTC)

Yes indeed... watchlisted both. Zad68 14:35, 1 May 2013 (UTC)
thanks! Jytdog (talk) 15:09, 1 May 2013 (UTC)

health review of effects of wine, in American Journal of Enology and Viticulture

An editor just added the following source to the article Health effects of wine. OK source under MEDRS, or not, what do you think? If it is, it would be the most recent acceptable review on the topic. Guilford J and Pezzuto J (2011) Wine and Health: A Review Am. J. Enol. Vitic. 62:4 (that would be American Journal of Enology and Viticulture) Jytdog (talk) 04:07, 5 May 2013 (UTC)

Red flags: Not MEDLINE indexed, health effects would appear to be outside the scope of the journal, stated as "enology and viticulture, including wine chemistry, sensory science, process engineering, wine quality assessments, microbiology, methods development, plant pathogenesis, diseases and pests of grape, rootstock and clonal evaluation, effect of field practices, and grape genetics and breeding". The area of effects of alcohol on health is not an area where research is so scarce we need to be reaching for suboptimal sources to use. I personally would avoid its use, though look through its citations for more appropriate sources to use. Yobol (talk) 12:27, 5 May 2013 (UTC)
I agree with this. I actually reverted the addition as being no good under MEDRS, but threw away the edit before I saved it because I was not certain myself and was very unsure if there would be consensus to keep this out of the article. Jytdog (talk) 14:38, 5 May 2013 (UTC)

Caprylidene / Axona... promotion?

I have twice deleted a large expansion of the Caprylidene article because it appeared to be based on poor sources such as the Axona brand drug maker's own guide to prescribing the drug, and because the initial author's username was clearly pushing Axona as a treatment for Alzheimer's. An IP editor quickly restored the material.

I am not very familiar with the ins and outs of MEDRS, so I would appreciate more eyes on the situation. Let me know if the sources are being used properly. Binksternet (talk) 00:42, 9 May 2013 (UTC)

Binks, thanks for bringing this. You very correctly identified the editor who added that, User:AxonaForAD, as having problematic promotional username. I saw you left a note on the User Talk page but it really should go to WP:UAA if the editor comes back.

As for the content of the large addition itself, there were many problems with it, but also some of it was really very useful. Only some of the content was actually sourced to the drug information sheet, the rest was sourced to some recent review articles (good), and a very old primary study (probably bad) in decent-looking journals. There was heavy use of this primary but it was a good-looking primary study, and the content used it appropriately. I don't know if there's a review article available that incorporates this study.

It seems really suspicious that this whole thing appeared as the editor's fist edits with no previous history, and it had a good general idea of what sourced Wikipedia article should look like. It was clearly written by someone with some clue. It needs to be Wikified and heavily copyedited to comply with WP:MEDMOS but it should not be thrown away. It would be a nice and worthwhile bit of work to take the content in that edit and bring it into Wikipedia standards. IMHO anyway. Zad68 02:44, 9 May 2013 (UTC)

At User talk:AxonaForAD I expressed my concern that the editor is being paid to improve the reputation of Axona, that the PR company brandkarma.org is at work on the article. I am not in favor of editors being paid to fluff articles, so I will leave the notional improvements to others. Binksternet (talk) 20:24, 9 May 2013 (UTC)
On the original inquiry, interesting question about what kind of source a drug label (what binkster calls the "prescribing information) is. I would be tempted to call it a secondary source, since a) it gathers information from many different studies and synthesizes them; b) it undergoes intensive review by the FDA before it is approved and published (probably more rigorous than peer review). And therefore, a drug label could be considered a solid secondary source under MEDRS. We don't want to copy/paste the thing of course, especially not all the prescribing information. I am curious what others think about what kind of source a drug label is. Correction, this stuff is medical food, not a drug. The label is not reviewed by the FDA like a drug label. I would say the label cannot be used except to quote it, like a primary source. And we should use the label as little as possible. Jytdog (talk) 21:29, 9 May 2013 (UTC)
(On the interesting question of how to classify a prescriber's insert, I think that there is a plausible case to be made for considering most of it to be a non-independent secondary source.) WhatamIdoing (talk) 01:42, 10 May 2013 (UTC)
WAID if we could get consensus on that characterization, it would be really useful info to add to WP:MEDRS. Zad68 01:50, 10 May 2013 (UTC)
On closer look, the content provided by User:AxonaForAD stinks, Binks was 100% right and I won't make this mistake again. Zad68 03:27, 10 May 2013 (UTC)
It didn't all stink! 1) there is a lot of legit work going on, pursuing metabolic dysfunction in AD brain and AxonaForAD did a good job of citing it. 2) Accera markets it as a medical food, not a drug. Not many people understand what "medical food" is and people make the mistake of treating it like a dietary supplement or a drug, when it is neither. However, in the "resurrection" of content I did, I did leave too much of the clean causal chain that Accera tries to draw, and I should not have reported the clinical trial results. Bad on me. Fixed both. Also added criticisms that zad found. Do let me know what you all think now. Jytdog (talk) 04:24, 10 May 2013 (UTC)
Article sounds great now. Really balanced. Great work. Have you ever red WP:BOOMERANG?--Garrondo (talk) 07:17, 10 May 2013 (UTC)
I have finally copied all the original content on glucose consumption (without mentioning axona) into Biochemistry of Alzheimer's disease, sincer content seemed valid, although it was used to make a POV in the axona article.--Garrondo (talk) 08:54, 10 May 2013 (UTC)
Glad you like it, garrondo. not sure why you reference boomerang - there was no ugly dispute. I think we worked together collegially to fix a bad article quickly, and zad made a great save there. Jytdog (talk) 12:04, 10 May 2013 (UTC)
I was referring to what can occur when you pay somebody to promote your company, and then this leads at the end to the addition by regular editors of content that may be negative for the company... which I feel it is what occurred here. Bests--Garrondo (talk) 13:36, 10 May 2013 (UTC)
👍 Like Good result from a questionable start. Zad68 13:45, 10 May 2013 (UTC)
Thumbs up icon Much better now Jyt. I was mostly embarrassed for myself for not catching the hugely COI study and wanted to be sure the article didn't hang out for too long with that study featured so much, but you jumped right on it and fixed it. Zad68 13:47, 10 May 2013 (UTC)

Glad you are OK with it zad, thanks again. And I hear you garrondo - thx for explaining! Jytdog (talk) 14:38, 10 May 2013 (UTC)

Neuroaid

is marketed as a stroke treatment. I have cleaned-up the article which seem to have a lot of poorly-sourced content - but medicine is not my core area so this could probably do with some wise eyes. Is this the right place to ask for help like this? Or is there a MEDRS noticeboard or the like? Alexbrn talk|contribs|COI 18:03, 13 May 2013 (UTC)

WT:MED is the most effective place to request help for problems like this. WhatamIdoing (talk) 23:35, 16 May 2013 (UTC)

Primary sources

I'm proposing to change this paragraph (lets reach consensus):

"ll Wikipedia articles should be based on reliable, published secondary sources. Reliable primary sources may occasionally be used with care as an adjunct to the secondary literature, but there remains potential for misuse. For that reason, edits that rely on primary sources should only describe the conclusions of the source, and should describe these findings clearly so the edit can be checked by editors with no specialist knowledge. In particular, this description should follow closely to the interpretation of the data given by the authors or by other reliable secondary sources. Primary sources should not be cited in support of a conclusion that is not clearly made by the authors or by reliable secondary sources, as defined above (see: Wikipedia:No original research). When citing primary sources, particular care must be taken to adhere to Wikipedia's undue weight policy. Secondary sources should be used to determine due weight."

To:

"ll Wikipedia articles should be based on reliable, published secondary sources. Reliable primary sources may be used with care. Edits that rely on primary sources should only describe the conclusions of the source, and should describe these findings clearly so the edit can be checked by editors with no specialist knowledge. Primary sources should not be cited in support of a conclusion that is not clearly made by the authors, (see: Wikipedia:No original research). When citing primary sources, particular care must be taken to adhere to Wikipedia's undue weight policy. "

Ryanspir (talk) 09:53, 7 April 2013 (UTC)

Strike throughs added by me for clarity. -- Brangifer (talk) 22:49, 7 April 2013 (UTC)
  • Completely Disagree: it leaves too much space to cite primary sources at the discretion of editors. I believe it is a change to the worse. I am fully comfortable with what we have right now.--Garrondo (talk) 15:15, 7 April 2013 (UTC)
  • Oppose There are lot of secondary sources that still need summarizing. The use of primary sources on Wikipedia generally represents an attempt to push something beyond its due weight. Doc James (talk · contribs · email) (if I write on your page reply on mine) 16:36, 7 April 2013 (UTC)
  • Strong oppose. We have enough trouble trying to explain why primary studies are usually inappropriate, this will just make things worse. Yobol (talk) 17:08, 7 April 2013 (UTC)
  • Strong oppose The "conclusion" of a primary research paper is nearly always a plea meaning "We've found something interesting/useful please continue to fund our research in this area / please sign the FDA documents that all allow us to sell it". They aren't neutral assessments at all. Colin°Talk 19:19, 7 April 2013 (UTC)
  • Strong oppose this actually cuts the heart out of MEDRS as a stronger statement of WP:PSTS and comes very close to violating WP:PSTS. The words "may be used as an adjunct to the secondary literature" are absolutely crucial and are far too often ignored. But thank you for crystallizing the issue with this clear statement.Jytdog (talk) 20:21, 7 April 2013 (UTC
  • Oppose We rely on independent expert reviews, textbooks, and other secondary sources to evaluate and contextualise primary research. We don't do that ourselves. --Anthonyhcole (talk · contribs · email) 21:03, 7 April 2013 (UTC)
I only ever use a primary source if it has first been put into context by a reviewer with the relevant expertise in a reliable independent secondary source. --Anthonyhcole (talk · contribs · email) 17:30, 12 April 2013 (UTC)
  • Oppose This section of WP:MEDRS as it currently stands is the proper application of Wikipedia sourcing policy to medical topics, the proposed change would disimprove the guideline. Zad68 22:05, 7 April 2013 (UTC)
  • Strong oppose. This proposal guts the guideline of important content. It goes way too far. -- Brangifer (talk) 22:49, 7 April 2013 (UTC)
  • Strong oppose. Current wording is better.Desoto10 (talk) 01:18, 8 April 2013 (UTC)
  • Oppose Current wording is better. But MEDRS needs to address the occasional case where a primary source is such a whale of a study that it wags all meta-analysis down the road. Examples: Women's Health Initiative and Million Women Study. These studies have their own WP articles, so we can hardly refuse to use their primary sources, er... primarily. Secondary sources just requote the primary conclusions of whale studies. We are left in the same position as when we get data from a big study with the Hubble Space Telescope or Large Hadron Collider which isn't likely to be repeated soon. You just have to bite your tongue and quote it, noting that it's a single LARGE study that hasn't been confirmed and probably won't be, for some time. That sucks, but meanwhile what can you do? SBHarris 02:38, 8 April 2013 (UTC)
Agree that this is the exception. Doc James (talk · contribs · email) (if I write on your page reply on mine) 02:42, 8 April 2013 (UTC)
Agree. This is the type of situation the guideline allows for. Maybe it needs to be made explicit, with these as examples? -- Brangifer (talk) 03:21, 8 April 2013 (UTC)
(Sbharris) The primary-study / secondary source debate for sourcing wikipedia really has nothing to do with the power of the study. That science may want to repeat small or surprising studies to prove they weren't a one-off, or not want to repeat large conclusive studies or studies that there simply isn't enough money/interest to repeat, is a matter for our secondary sources and the academic/scientific community to decide. I don't see these as an exception to our rules. I'm sure both the studies you mentioned were covered rapidly by secondary sources of various kinds. Colin°Talk 07:04, 8 April 2013 (UTC)
It usually does not take long before these major studies are parts of reviews. Doc James (talk · contribs · email) (if I write on your page reply on mine) 05:21, 9 April 2013 (UTC)
Of course, but I'm not sure it improves them. If there's just one big study of something, any "review" that uses it (cites it) is forced to conflate it with a lot of stuff that is only semi-confirmatory, since the extra stuff used to compare it with, won't be the same thing. For example, a big giant prospective study, the only one of its kind, might be conflated with some other really big cohort-controlled prospective epidemiological study. And that's actually exactly what happened with the two studies above, that I cited. They aren't the same-- in one study the groups are split at random, and in the other, the groups choose themselves and (of course) therefore aren't the same. So what do you do when the findings from these studies turn out the same, or when the data conflict? Which they do at many points (sometimes they agree and other times they don't). We simply end up playing off one review against another, and in that case, one would need a review of reviews, or a review of the quality of previous reviews. But suppose there's only ONE of those? How do we know it's not biased? Of course it's biased. There is no getting away from the problem that one must cut off the generalization at some level, and sometimes a lower level is appropriate is there's only one event or one big study that is relevent. In that case, the most appropriate reviewers of a set of data, are the people who generated it. Everybody else down the line is simply using data they got from somebody else and doing extra induction and comparison with similar studies, that the primary authors chose not to do.

At the most ridiculous level, we have a gigantic multicenter study that appears in the New England Journal of Medicine, peer reviewed by the best in the field, and we can't use it until somebody in the Iowa Journal of Family Practice Reviews notices the thing. Then we cite THEM. Meanwhile, why doesn't all the reviewing that contributing to getting it into the NEJM in the first place, count? It's all very arbitrary, seems to me.

Often it takes time and effort, and you must survive more scrutiny, to get some data into an exclusive high-quality primary journal, than it takes to get it they same paper into a third-tier primary one AND then reviewed by several others.SBHarris 03:10, 30 April 2013 (UTC)

We don't require a meta-analysis or some other combination with other studies. A narrative review that says little more than "This awesome study clearly proves that the foo is bar baz during full moons" is good enough for our purposes. What we're looking for is essentially an expert endorsement of someone else's work, rather than Wikipedia editors deciding which experiments were well done. WhatamIdoing (talk) 03:35, 30 April 2013 (UTC)
The whole idea of peer review is that some (bunch of) experts have to endorse a study as well-done and belieable. [Though not necessarily considered right until checked by one-or-more independent good studies elsewhere]. Anything the NEJM publishes (for example) will be picked up by secondary review journals, authomatically. And I mean automatically. It's not that some expert decides it's a great study. That's considered a given. if it even appears in the NEJM in the first place. So John Q. Editor at WP can do that as well as anybody. The secondary review journal doing that, doesn't add a thing to the credibility of the paper.

For example, today's NEJM has an article addressing a previous NEJM report of the antibiotic azithromycin causing heart deaths, based on data from Tennessee. This study had been picked up many secondary sources, including newspapers, and even caused the FDA to issue a boxed warning. Alas, none of these were any better than the first study, though they did cause the problem to make it into the Wikipedia article on the drug: [4]. Today's new study finds the supposed safety problem is not seen in a slightly different population from Denmark. So it wasn't rotten in Denmark. Now we dither, and even the FDA dithers on prescribing: to warn, or not to warn, that is the question. But no medical journals have picked up the recent finding, yet. Why should we wait till they do? They won't be doing it on the basis of anything we don't already know. They will be automatic. When they quote the new NEJM, as they inevitably will, they don't add anything by doing so. So why are we waiting for them? It's the NEJM. We don't need somebody else to read it to us. We didn't need it the first time. SBHarris 03:33, 1 May 2013 (UTC)

I disagree with your "whole idea of peer review" opinion and certainly with your view that anything the NEJM publishes can be used directly by wikipedians as though it is God's own truth. I don't think you really appreciate the WP:WEIGHT issues of new research and perhaps have got Wikipedia confused with a newspaper in terms of WP:RECENTISM. As far as Wikipedia is concerned the writers of the "Iowa Journal of Family Practice Reviews" are way ahead of Sbharris or any other Wikipedian when it comes to assessing the primary research literature. There will always be editors who think they can review the primary research literature better than the review and textbook authors we are supposed to rely on. If you feel that way, write a blog or get yourself properly published, because that isn't your role as a Wikipedian. Colin°Talk 15:11, 1 May 2013 (UTC)
I agree with Colin above. Sbharris, there is a really important reason that both 1) the fundamental policy WP:OR and its section on primary/secondary/tertiary sources WP:PSTS, and 2) MEDRS (which is completely within PSTS), each warn very strongly away from basing content on primary sources, and that is due to the dangers of cherry-picking, SYN, and WEIGHT. As Colin said we editors need outside experts to publish a review that discusses the great NEJM study, putting it in its proper context. We need that 2ndary source to get the content in, and to defend it against people who would take the content out. While you may believe yourself to be educated/experienced enough to provide that context yourself, you could not defend the content providing the context on your own authority as an editor and editors cannot make those kind of judgements (think of the disasters that would happen if the policy were not this way, and people who thought they knew what they were talking about, but didn't, were making these kind of judgements throughout wikipedia). I realize that you find the secondary source policy frustrating and too slow and maybe too stupid, but it is a good and robust policy that is essential for Wikipedia to be the wide-open encylopedia that it is. Happily, landmark clinical studies are often accompanied by a comment letter by the editors of the journal or some thought-leader, which provides context and describes the importance of the study -- that comment letter can serve as the 2ndary source to get the Important Study into wikipedia. Jytdog (talk) 15:43, 1 May 2013 (UTC)
Letters to the editor, comments, editorials, and other opinion pieces are never peer-reviewed and are treated as primary sources on Wikipedia. They consequently aren't capable of getting the Important Study into Wikipedia. WhatamIdoing (talk) 16:25, 1 May 2013 (UTC)
That is interesting, and a good point. Thanks. Jytdog (talk) 18:22, 1 May 2013 (UTC)
I would like to point out that the devil is not as scary as he looks :). That's a Russian proverb, not sure if it makes sense in English though.
In my proposal I have indicated that:
1. "Wikipedia articles should be based on reliable, published secondary sources"
2. That only *reliable* primary sources can be used and when so is done, it should be done with care.
3. "Primary sources should not be cited in support of a conclusion that is not clearly made by the authors, (see: Wikipedia:No original research)." - I have left it as it is.
4. "When citing primary sources, particular care must be taken to adhere to Wikipedia's undue weight policy." - I have left this limitation as well in my proposal.
5. Regarding primary sources that are being adjunct to secondary sources, there is no need IMHO, as we have already mentioned that primary source should be used with care and put provided two limitations on its use. In any case, it's the consensus on every article that will be the deciding factor in case any discussion will arise if any editor has used a primary source "with care" and it will be disagreed by another editor. I repeat again, my version is more clear, shorter and more straight forward than the original and it gives just *slight ly* more air for inclusion of primary sources while including the limitations. Ryanspir (talk) 14:44, 11 April 2013 (UTC)
  • Oppose - This is a move in the wrong direction. If anything, we need to tighten the loophole to ensure that primary sources are not treated as reliable unless secondary sources have first cited them in a way that shows them to be credible (i.e. the citation is not just to make a criticism, update, or retraction), significant (i.e. the secondary source uses the main findings from the primary source), and independent (i.e. no tie between the authors of the primary and of the secondary source). At that point, the utility of the primary source is simply to fill in some blanks that the secondary source omits. LeadSongDog come howl! 15:12, 11 April 2013 (UTC)
  • Are you assuming that editors must be "enforced" and saying "use with care" is not adequate? Ryanspir (talk) 15:56, 11 April 2013 (UTC)
  • This would simplify things enormously but it would overly constrain us when a study appears addressing a rarely-reviewed uncontroversial topic - I gather quite a bit of biology is slow to be independently reviewed. Though, in those very, very rare instances in health-related topics I'd prefer to see editors simply reporting the results and all the caveats but none of the usual evaluation, contextualisation, speculation or puffery found in the "conclusions" section, and making it clear somehow that the results have not been independently reviewed yet. ("In a recent placebo-controlled trial, blah blah, though these results are yet to be subjected to independent scholarly review" or something of the sort.) --Anthonyhcole (talk · contribs · email) 01:44, 13 April 2013 (UTC)
I agree with the position of Anthonyhcole. The example of the way how the attribution should be made in such cases can be given in WP:MEDRS. Ryanspir (talk) 13:03, 13 April 2013 (UTC)
I think the present wording of the guideline is clear enough. Just to emphasise: I've never, to my knowledge, added a health-related assertion to an article based on an un-reviewed report. And I've only seen it done appropriately a handful of times over the last five years. In most other cases the editor either doesn't understand the principle that Wikipedia simply reflects significant expert views, they disagree with that principle, or they're pushing a discredited or fringe view. --Anthonyhcole (talk · contribs · email) 03:48, 21 April 2013 (UTC)
Due to a strong consensus, may I kindly propose to remove primary sources "LA article" and "Quackwatch" on the article Medical uses of silver? Calling a substance "quack" implies that the substance is not effective for the conditions mentioned, and calling the substance "quack" in general implies that it is not effective for any conditions. Per the strong consensus in this section such determination by a primary source should have an adjunct reliable secondary source in order to comply with WP:MEDRS guidelines. Thank you. Ryanspir (talk) 09:13, 22 April 2013 (UTC)
This is so far from prior statements on this topic that it's hard to understand why you'd try this again. -- Scray (talk) 14:25, 22 April 2013 (UTC)
No. Zad68 14:30, 22 April 2013 (UTC)
No: Quack is a social term with a broader meaning that simple effectiveness. Quack is not even a medical term and proof is that search for quack in pubmed gives few results (almost all from authors with that name). In this sense it implies that there is some consensus outside the medical field to consider a treatment and its commercialization a fraud. Such consensus may be right or wrong and to indicate that we should provide secondary sources on efficacy, but if a reliable social source (such as NYT) says something is quack that clearly merits inclusion in an article since it implies that society in general does not consider the claims reliable (independently of truth). Importance is in its social aspects and that is why no secondary source is needed, since it is not a medical claim. --Garrondo (talk) 14:54, 22 April 2013 (UTC)
Ryan has a long history of making edits that conflate the substance of colloidal silver itself with the "quack" uses of it, and also of making edits confusing primary and secondary sources. Zad68 14:58, 22 April 2013 (UTC)
May I kindly bring your attention to: "Quackwatch states that colloidal silver dietary supplements have not been found safe or effective for the treatment of any condition." It clearly says "not been found effective" "for the treatment of any condition". Such statement should in my opinion have adjunct reliable secondary source. It is a medical claim, right? Ryanspir (talk) 20:27, 24 April 2013 (UTC)
Quackwatch is reporting on what the FDA has said. The FDA has stated that cs dietary supplements have not been found to be safe or effective.Desoto10 (talk) 00:28, 25 April 2013 (UTC)
FDA is not listed in WP:MEDRS as an option to which a primary source can be adjunct to. Only secondary sources are listed. Thank you. Ryanspir (talk) 01:56, 26 April 2013 (UTC)
Just a suggestion: sometimes the best way to deal with WP:IDHT is to WP:SHUN the problem. Yobol (talk) 00:54, 25 April 2013 (UTC)
I'm acting in good faith and what I'm asking is to impartially implement the strong consensus we have reached here towards primary sources. I don't think your WP:SHUN is warranted, as I'm being very reasonable and listening to everyone's opinion. Ryanspir (talk) 10:05, 26 April 2013 (UTC)
  • Support proposal. I said the same thing myself before. But note that you're up against a powerful lobby here, funded via Wiki Med Inc., so this is a waste of time. Wnt (talk) 18:18, 29 April 2013 (UTC)
    Now I understand: there's a well-financed conspiracy, and all these people advocating secondary sourcing are making money from it? You're way ahead of me on this. -- Scray (talk) 00:11, 30 April 2013 (UTC)
    That's one of the daftest things I've heard in ages. Wiki Med Foundation (not Incorporated) is a charity and has next to no money. There is also no reason why the Foundation would wish the extermination of primary sources. JFW | T@lk 19:01, 17 June 2013 (UTC)
  • Comment - I am fine with this part staying as it is, however I do notice that it gets misinterpreted. "In particular, this description should follow closely to the interpretation of the data given by the authors or by other reliable secondary sources." This point is covered completely in the sentence after it, and it only adds confusion. Primary sources should stay true to their own interpretation and not be dependent on another source to interpret it, that part is highly susceptible to misinterpretation. The last sentence also gets misinterpreted, "due weight".While a secondary source can critique/review/opinionate (interpret in these senses of the word) a primary source, in no way should a second source even by omission tell a primary source what it meant to say. Those two sentences are very unclear, messy, and open to misinterpretation.
I did notice the edit made by user:Wnt in the guideline article, and I agree with that part. Primary sources are important especially when they are up to date, reliable, and offer vast amounts of knowledge. Editors shouldn't be so quick to discount primary sources, when they are allowed. Strict usage of primary sources is acceptable, but it is not acceptable to discount them. Sidelight12 Talk 20:11, 19 May 2013 (UTC)
  • Oppose. As others have pointed out, we need a policy that discourages the use of primary sources in the strongest possible terms. Where primary sources are used (on editorial judgement) they need to be supported by secondary sources as much as possible. It is most uncommon for a primary source to displace the conclusions of secondary sources, and usually you need a secondary source to support the claim that a primary source displaces the previous secondary source (yada yada). If every single health- & medicine-related editor would adhere to the simple policies on secondary sources we'd be able to avoid about 60% of all disputes. JFW | T@lk 19:01, 17 June 2013 (UTC)

Reliability of position statements from nationally or internationally recognised expert bodies

1) Sodium:

"Although the advice to restrict sodium to 1,500 milligrams a day has been enshrined in dietary guidelines, it never came from research on health outcomes, Dr. Strom said."

2) Vitamin D:

"Because of the British experience with idiopathic infantile hypercalcemia attributed to hypervitaminosis D, an inaccurate association occurred that had a profound effect on the potential of vitamin D supplementation, not only during infancy but also during pregnancy. In 1963, Black and Bonham-Carter (65) recognized that elfin facies observed in patients with severe idiopathic infantile hypercalcemia resembled the peculiar facies observed in patients with supravalvular aortic stenosis syndrome. By 1966 vitamin D was viewed by the medical community as the cause of SAS syndrome. (66,67) With the advent of molecular genetics, the children with SAS Syndrome were discovered to have Williams Syndrome, an example of unipaternal disomy, with abnormal vitamin D metabolism. (68–75)

The perception that vitamin D can inflict harm during pregnancy still lives on today as many obstetrical specialists are afraid to undertake vitamin D repletion during this period. Research efforts in this area were further hampered when in 1997, the Institute of Medicine issued guidelines that defined the adequate intake (AI) for vitamin D during pregnancy to be 200 IU/d with intakes greater than 2000 IU/d causing potential harm. (40) Recently, the IOM issued new guidelines with respect to pregnant women that define the estimated average requirement (EAR) and recommended dietary allowance (RDA) to be 400 and 600 IU/day, respectively. They also increased the tolerable upper intake limit (UL) to 4000 IU/day. (62)These new guidelines, with the exception of the UL, are based on old data since limited new data exist. The result of prior and current guidelines is that most prenatal vitamins only contain 400 IU of vitamin D. In our experience, many of today’s practicing obstetricians are unaware of the vitamin D content in prenatal vitamins or have a fear of administering additional vitamin D supplements to the pregnant women."

3) Radiation:

"In 1958, using data largely drawn from these bomb studies, scientists came up with an answer. It was called the Linear No Threshold (LNT) model and suggested all radiation, no matter how small, was dangerous.

It became the internationally recognised basis for assessing radiation risk. Yet there has always been a problem with it. The data from Hiroshima and Nagasaki were for very high levels of radiation exposure, often in the range of thousands of millisieverts. There were no significant data for lower exposures, particularly below 200 millisieverts.

The model was based on high doses and we just didn't know what was going on at lower doses of between one and 200 millisieverts," says Dr Repacholi.

Scientists simply guessed that if high-level radiation was dangerous then lower levels would also be hazardous. They made "an assumption", observes Dr Repacholi.

Chernobyl, where most people received radiation doses below 200 millisieverts, has been the first large-scale opportunity to test whether this assumption is true. The evidence from the Chernobyl Forum suggests it is not.

"Low doses of radiation are a [very] poor carcinogen," says Professor Brooks, who has spent 30 years studying the link between radiation and cancer."


"Professor Ron Chesser, of Texas Tech University, US, has spent 10 years studying animals living within the 30km exclusion zone surrounding Chernobyl.

He has found that, far from the effects of low-level radiation being carcinogenic, it appears to boost those genes that protect us against cancer.

"One of the thoughts that comes out of this is that prior exposure to low levels of radiation actually may have a beneficial effect," Professor Chesser says."


"In April, the WHO's International Agency for Research on Cancer (IARC) published a report that used the latest LNT-based radiation risk projection models to update the estimated cancer deaths from Chernobyl.

It concluded that about 16,000 people across Europe could die as a result of the accident."

Count Iblis (talk) 18:21, 14 May 2013 (UTC)

I'm sorry; I've read the above, but I'm not sure what your point is. Do you mean to illustrate that scientists and expert bodies are fallible and sometimes proven wrong with the passing of time? Or that they are sometimes forced to extrapolate and make educated guesses where evidence is lacking? Or that individual scientists sometimes disagree with the views of expert bodies? Help me out. MastCell Talk 18:43, 14 May 2013 (UTC)
It seems to me somewhere along the chain of primary resesearch, secondary reviews etc., all the way up to the highest authoritative expert bodies, unscientific arguments gets injected in a way that is not typical in other sciences. Of course, unlike in other sciences, in medicine expert bodies may need to make a judgement calls when the evidence is not clear, but it seems to me that such a judgement call will persist long after the science has become clear. So, in case of 2) pregnant women are still not getting the vitamin D supplementation they need, even though decades have passed since the original worry has been clarified. Count Iblis (talk) 18:59, 14 May 2013 (UTC)
Is this a suggestion to expand our content at Vitamin D? I peeked at that article and it does say that Vitamin D deficiency during pregnancy is harmful and that women often do not get enough, although it also says that supplementation is of unknown value. I do not see the source you provided in use at that article. Maybe make those edits?

Like MastCell I'm not sure what the larger point is here. Yes sometimes expert bodies get things wrong and it's hard to change entrenched positions. Still using secondary sources from expert bodies is MUCH more likely to be providing good information than using the latest individual primary sources, whatever they happen to say and of whatever quality they happen to be, or was that even the underlying point? Is there a change to WP:MEDRS being proposed? Sorry still not seeing what we're getting at here at WT:MEDRS. Zad68 19:42, 14 May 2013 (UTC)

I'm raising a point about this policy text which is written from the assumption that the highest medical expert bodies stick to science in the same was similar expert bodies in other sciences. This is not the case because these expert bodies have a dual role. They don't just review the science, they also have to come up with guidelines for doctors and patients. They need to weigh up the scientific results as published in secondary review articles based on an appropriate precautionary principle. The conclusions they reach are then not going to be conssistent with what you would get if you were to completely neutrally weigh up the scientific results. This can lead to problems that I cited above, but in general in the absense of these problems, there will still be an issue to be addressed.
A good example is vitamin D. Good review articles on toxicity are this one that considers mainly the measured effects and this one that considers it from the POV of the fundamental biological processes. But what we do in the Wiki articles is take more serious the IoM report which puts a lot of weight on a single primary reseach result which wasn't all that statistically significant, where it was found that elderly patients in some elderly care home fell more often if they were given vitamin D supplements at levels. What the IoM does here is quite obvious, in the absense of very strong evidence of health benefits (other than for bone health), you need to rule out with almost 100% certainty that there could be problems with vitamin D supplementation. So, even questionable research results hinting at problems have to be taken serious, while for evidence purporting to show benefits you would demand a very rigorous level of proof.
The problem for Wikipedia is then that with the current policy text, you can't address this issue of improper weight. E.g. in the literature you can find discussions about why these elderly people would fall more often (e.g. there is quite strong evidence that vitamin D improves neuro-muscular function, the people in non-placebo group will get out of their chairs more often and therefore be at a greater risk of falling), but these are secondary review articles and are given less weight than the IoM report, despite having a more fundamentally scientific focus on this issue. Count Iblis (talk) 14:01, 16 May 2013 (UTC)
We are not, in fact, assuming that the highest medical expert bodies stick to science. We are instead assuming that practice guidelines contain significant material about what the mainstream expert opinion is and what the actual practices are, both of which are relevant, important, and must receive significant weight to create a neutral article.
You may be confusing "neutral" with "correct". For better or worse, Wikipedia does not aim to write scientifically ideal articles. It instead aims to write encyclopedia articles that adequately and accurately reflect the current mainstream expert opinion, along with any significant minority viewpoints. If, as does happen, the experts are wrong (or divided), then it is the English Wikipedia's official policy that we, too, will be exactly as wrong (or divided) as they are. WhatamIdoing (talk) 23:42, 16 May 2013 (UTC)

Seralini Thai study on effect of glyphosate on human breast tissue, with older in vitro study from Seralini group

Note - edited the title just now as per semitransgenic's note below, with italics showing added text and crossout showing deletion.Jytdog (talk) 17:01, 16 June 2013 (UTC)

Three efforts have been made to include content in the glyphosate article based on recent primary study on the effect of glyphosate on human breast tissue from a Thai group, 'the last of which was accompanied by a 2009 in vitro study by the Seralini group Both User:Zad68 and I have reverted each time - last one is here as an example of the kind of content being created. Each time, the content has been created in the section on the Toxicity of glyphosate to humans - -indeed the only reason to include this study in the article is the effect on human health, and I believe that including it goes against MEDRS. Folks are objecting on Talk here so I wanted to get feedback on whether MEDRS applies to content generated from this source and if so, what kind of content, if any, is appropriate to generate from it. Thanks. Jytdog (talk) 22:53, 15 June 2013 (UTC) (edit note -- added the text in italics as per semitransgenic's note below. Jytdog (talk) 17:01, 16 June 2013 (UTC))

Yes would be appreciated to have more of those familiar with the application of WP:MEDRS commenting. Zad68 03:24, 16 June 2013 (UTC)
Primary sources can be included. You're right that it cannot pertain to humans (section: Avoid over-emphasizing single studies, particularly in vitro or animal studies). It says preclinical studies are invaluable for scientific research. I think it should be kept, but any mention of what it may imply for human benefit must to be deleted. Lucy346 (talk) 06:55, 16 June 2013 (UTC)
As pointed out by Zad, this source has intrinsic implications for human health- that is how it is written. Am I reading the same policy? Primary sources are strongly discouraged, and their use can only be potentially justified in very specific circumstances. This particular paper, by virtue of being in vitro is not suitable. It is very clear: the policy says, when in doubt- do not include the primary source. Lesion (talk) 08:50, 16 June 2013 (UTC)
The section "Avoid over-emphasizing single studies, particularly in vitro or animal studies" has to do with this. I don't see where it says, when in doubt, don't use the primary source. If this study is in vitro, and it assumes these results as benefits for humans, rather than potential harmful effects, then that is a problem. That part can't be used, but it doesn't necessarily make the source completely unusable. The abstract says, "potential adverse health effects," so I don't know if that makes it ok. If this stays, its dangers need to be documented too. Lucy346 (talk) 09:47, 16 June 2013 (UTC)
it's a mischaracterisation to call this a "Seralini study" in the heading, and it's intentionally misleading. But whatever, that's the kind of dishonesty we are dealing with here. Guideline does not state primary sources cannot be used, it cautions against overemphasis, it's a peer reviewed study published by a reputable journal, appropriately weighted mention, in the correct context, is acceptable. Semitransgenic talk. 16:24, 16 June 2013 (UTC)
Good point, and thank for pointing out the conflation. It wasn't intentional, I was just moving too fast. I lumped the 2013 primary source from the Thai group with the 2009 primary source from the Seralini group into one discussion; both were included in the last attempt to add content to the article, as you can see in the dif I referenced at the beginning. Both have the same problem, in that they are primary sources and present results of in vitro studies, and as such should not be used to discuss health. Jytdog (talk) 17:01, 16 June 2013 (UTC)
When you click the bottom of "related articles" in pubmed it yields a 100 related primary studies to this one. Secondary sources provide the background to decide which content is important and which is not. To cite this specific article, and not the other 100 is by itself to give undue weight since we do not have any reason to believe that it is more important than any other one.--Garrondo (talk) 19:38, 16 June 2013 (UTC)
"To cite this specific article, and not the other 100 is by itself to give undue weight since we do not have any reason to believe that it is more important than any other one." Completely false logic, you are implying that there are 100 similar studies concerning the role of glyphosate as a potential endocrine disruptor. The relevance of the study in question (in the context of "glyphosate toxicity") is pretty clear. Semitransgenic talk. 22:45, 16 June 2013 (UTC)
Secondary sources exist and there is no reason why we need to use primary sources. Wikipedia is not a collection of everything written. Using secondary sources helps use give proper weight. Doc James (talk · contribs · email) (if I write on your page reply on mine)

Here's an example of the text that's been disputed:

A June, 2013 article published in a peer-reviewed journal entitled: "Glyphosate Induces Human Breast Cancer Cells Growth Via Estrogen Receptors" suggests that low and environmentally relevant concentrations of glyphosate possess estrogenic activity and noted that several recent studies demonstrated glyphosate's potential adverse health effects to humans concluding that it may be an endocrine disruptor.[5][6]

Okay: The first bit is a problem per WP:MEDMOS. See all that stuff about hyping a study by claiming that it's "peer-reviewed", etc. So let's de-fluff it, in which case we get this:

One source says that low and environmentally relevant concentrations of glyphosate possess estrogenic activity. One source said that several recent studies demonstrated glyphosate's potential adverse health effects to humans concluding that it may be an endocrine disruptor.[7][8]

Now we have to process the content. Rather than adding a bunch of qualifiers like "one source that looked only at isolated cell lines, which are known to be a bit odd", let's see if we can deal with the big problems:

  • For the first sentence, who really cares what just one source said? PubMed gives me more than 1500 hits on glyphosate. Why not pick one of the dozens of review articles instead of this one? This doesn't seem like such a rare opinion that we need to risk looking like we're cherry-picking.
  • For the second sentence, you are presenting analytical conclusions rather than descriptions. A secondary source is required for this kind of claim. Again, these aren't exactly unusual claims in this area, so why aren't you citing a stronger source already? WhatamIdoing (talk) 23:06, 16 June 2013 (UTC)
As answer to Completely false logic, you are implying that there are 100 similar studies concerning the role of glyphosate as a potential endocrine disruptor. The relevance of the study in question (in the context of "glyphosate toxicity") is pretty clear.: says who? Several editors have already commented that they do not really see its relevancy. --Garrondo (talk) 07:58, 17 June 2013 (UTC)
Moreover, as stated above, to use it to back up results by others is highly innaproppiate, since it would be to use it as a secondary source. Similarly, the "envirommentally relevant" line is the conclussion from the authors but, by conclussions from primary articles are very commonly quite hyphenated so as to make the article more interesting". Content that would have to be included would be something like Glyphosate exerted proliferative effects only in human hormone-dependent breast cancer, T47D cells, but not in hormoneindependent breast cancer, MDA-MB231 cells, at 10-12 to 10-6 M in estrogen withdrawal condition., which is the true result from the article, and IMO absolutely undue weight. --Garrondo (talk) 07:58, 17 June 2013 (UTC)

Advice sought

I removed this section from Aloe arborescens on the grounds that it made claims not supported to the standards of WP:MEDRS, although there was some material inside the section which might possibly have been ok if carefully re-written.

Another editor has since re-written the material here, which seems to me not to make claims about effectiveness in human use, and so is ok. However, I'd like to know what colleagues here think. Peter coxhead (talk) 10:45, 16 June 2013 (UTC)

Neither source there should be used. Both are primary studies and so including them violates WP:PSTS as well as WP:MEDRS. If you search pubmed as follows [9] you get three reviews in English, two of which would be useful - one very useful for tox in cosmetic ingredients http://www.ncbi.nlm.nih.gov/pubmed/17613130 and the other which may be useful for an overview of potential dietary supplement or cosmetic ingredient uses as well as research on potential medical uses http://www.ncbi.nlm.nih.gov/pubmed/10624859 Jytdog (talk) 13:26, 16 June 2013 (UTC)
I think this is (barely) okay, since it sticks to a plain description. On stylistic grounds, I might prefer to see it condensed. But why not add the better sources yourself? WhatamIdoing (talk) 23:09, 16 June 2013 (UTC)
WP:PSTS doesn't say primary sources aren't allowed. It says be careful not to make conclusions based on primary or other sources. - Sidelight12 Talk 03:29, 17 June 2013 (UTC)
WhatamIdoing, you are forgetting WP:WEIGHT, which is far and away the most important reason not to use a primary research paper as a source. A five-year-old study on rats can't warrant a full paragraph of text sourced only on the primary research paper itself. Has this led to human-relevant trials or nowhere? What do secondary sources say? And the four-year-old cancer study must surely have drawn secondary commentary by now. So the question isn't whether the sources are reliable on what the article says but whether we should mention them at all, and in what detail. Whenever we mention basic (e.g. rat) research or clinical trials we imply this is relevant to the reader and has some importance to the subject. But Wikpedian's have no authority to decide which facts are relevant or important: we need to review the literature to find out. Colin°Talk 10:25, 17 June 2013 (UTC)

Clarification

"All Wikipedia articles should be based on reliable, published secondary sources," seems to contradict "Therefore, it is vital that the biomedical information in all types of articles be based on reliable, third-party, published sources and accurately reflect current medical knowledge." so it should be changed to "All Wikipedia article topics should be based on reliable, published secondary sources." Since this is a general guideline and not a medrs guideline. The general guideline is to use secondary sources to establish notability for the article topic.

"In particular, this description should follow closely the interpretation of the data given by the authors or by other reliable secondary sources." The word "other" only muddies the meaning. if you read it slowly, and if you read it fast, the word only changes the meaning. It should be removed. The word other implies a meaning that is not there if you take the time to read it slowly.

I moved this down to the advice section from the definition section, because these aren't definitions, they are guidelines.

I added a comma and that also got reverted. Its supposed to have a comma to be grammatically correct, and it tells the reader to slow down. The meaning of it seems to change if you read it fast, as opposed to reading it slowly with or without the comma.

My edits shouldn't have been reverted, it didn't change anything, only clarified this. Leaving these edits in is promoting the mistake of misreading it. - - Sidelight12 Talk 03:48, 17 June 2013 (UTC) / 06:45, 17 June 2013 (UTC)

I do not understand how the first two statements are supposedly conflicting. "All Wikipedia articles should be based on reliable, published secondary sources" refers to the actual content. The use of secondary sources is not merely limited to deciding whether to have an article about it ("notability"). Most of that content should come from secondary sources, such as review articles, university-level textbooks, and high-quality reference works. WhatamIdoing (talk) 06:17, 17 June 2013 (UTC)
I mixed up third-party with tertiary sources, so I crossed that part out. - Sidelight12 Talk 06:41, 17 June 2013 (UTC)
I disagree that your edits should not have been reverted - it is generally unwise to change a guideline or policy without discussing it first, as they are closely watched. That aside, I don't understand what you are getting at by adding the word "topics" - can you clarify? Do you maybe mean "content" (i.e. specific passages)? Moving the passage to Basic Advice as you propose makes sense to me. Jytdog (talk) 13:09, 17 June 2013 (UTC)
It says "All Wikipedia articles," which makes it a wikipedia wide policy and not a specific medrs policy. The Wikipedia wide policy is that secondary sources be used to establish the notability of an article topic for whether the article should exist, and not the content within the article. By article topic I mean the article namespace, which is already established, not the content within the article. I don't think there is a guideline anywhere that says, all content within all wikipedia articles should be solely based on reliable, published secondary sources. Rather, all content within all Wikipedia articles should be based on reliable published sources. - Sidelight12 Talk 04:27, 18 June 2013 (UTC)
You are completely wrong: While notability is indeed established by secondary sorces that is not in contradiction with the fact that wikipedia articles should be based in secondary sources. MEDRS is only the application to medicine of WP:NOR, which in a section (See: WP:PSTS) says
Wikipedia articles should be based on reliable, published secondary sources and, to a lesser extent, on tertiary sources. Secondary or tertiary sources are needed to establish the topic's notability and to avoid novel interpretations of primary sources, though primary sources are permitted if used carefully. All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source, rather than to an original analysis of the primary-source material by Wikipedia editors
and
Policy: Wikipedia articles usually rely on material from reliable secondary sources. Articles may make an analytic or evaluative claim only if that has been published by a reliable secondary source..
Similarly when choosing sources (See WP:RS) says
Articles should rely on secondary sources whenever possible. For example, a review article, monograph, or textbook is better than a primary research paper. When relying on primary sources, extreme caution is advised: Wikipedians should never interpret the content of primary sources for themselves.
and
Isolated studies are usually considered tentative and may change in the light of further academic research. The reliability of a single study depends on the field. Studies relating to complex and abstruse fields, such as medicine, are less definitive. Avoid undue weight when using single studies in such fields. Meta-analyses, textbooks, and scholarly review articles are preferred when available, so as to provide proper context
and
Primary sources are often difficult to use appropriately. While they can be both reliable and useful in certain situations, they must be used with caution in order to avoid original research. While specific facts may be taken from primary sources, secondary sources that present the same material are preferred. Large blocks of material based purely on primary sources should be avoided. All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source, rather than original analysis of the primary-source material by Wikipedia editors
and
Ideal sources for biomedical assertions include general or systematic reviews in reliable, third-party, published sources, such as reputable medical journals, widely recognised standard textbooks written by experts in a field, or medical guidelines and position statements from nationally or internationally reputable expert bodies..
In summary: Content in Wikipedia articles should be based in secondary sources to avoid original research. In the future please discuss changes to policies and guidelines first. --Garrondo (talk) 07:12, 18 June 2013 (UTC)

Vote for this change only, removal of the word "other" from the following sentence to:

"In particular, this description should follow closely the interpretation of the data given by the authors or by other reliable secondary sources."

*Support - nominator of proposal - Sidelight12 Talk 11:07, 19 June 2013 (UTC)

Proposed edit to guideline

Sorry gang I am back again with the toxicity issue, this time with a specific proposal.

Briefly, IMO there is a widespread problem on Wikipedia with regard to articles that discuss toxicology -- namely, one finds content on toxicity sourced from primary studies reporting results of in vitro and animal studies of a product or active ingredient, conducted by basic researchers with no training in toxicology. Too often articles are filled with laundry lists of such content. These sections are often under section headers like "Health Effects" and in my mind, their appearance in Wikipedia violates MEDRS and PSTS and -- and they serve just to scare people. Examples are Bisphenol A, Glyphosate, Endosulfan in subsections under "Health Effects" on endocrine disruption and developmental effects, The DDT article generally handles this very well but this problem has crept into discussion of "links" to developmental problems and cancer.

Toxicology is a discipline in which scientists are trained to consider typical routes and dose of exposure to a substance, design in vitro and animal experiments to obtain useful data, and then to extrapolate those results to humans. It is special training that a typical doctor or basic biomedical researcher does not have; regulators rely on toxicologists to critically review marketing applications (and to review subsequent studies to decide whether a product should be taken off the market) and companies rely on them to decide whether to invest money in further development. Wikipedia should also rely on them for sources about toxicity, in order to ensure that we provide reliable, consensus content.

To help manage this better, I'm proposing that the following be added to MEDRS, perhaps in the "Assess evidence quality" section, as a new last paragraph. WIth respect to toxicity, that section is especially problematic because it was not written with toxicity in mind (e.g. the paragraph that starts with "The best evidence comes primarily from meta-analyses of randomized controlled trials (RCTs)..." doesn't provide guidance for dealing with toxicity, as it directed to interventions like drugs or medical devices)

I am very open to suggested changes - here it is:

"Content discussing the toxicity of substances falls within this guideline. The toxicity of drugs is studied in cells, animal models, and in RCTs during drug development, and may also emerge in post-marketing surveillance. Substances that are not intended to benefit health ("non-drug substances") will never be the subject of an RCT, since such trials would be unethical. Observational studies of the effects of non-drug substances on humans, or the presence of non-drug substances in humans, may be conducted and published after a product comes to market; these are primary studies. Toxicity studies are generally carried out in animal models or cells, and the results are extrapolated to humans; these too are primary studies. Content that discusses the toxicity of non-drug substances should be sourced from secondary or tertiary sources written by toxicologists, in which primary sources are critically reviewed for their relevance to humans under typical exposures and doses. Primary studies should not be used to support content that discusses toxicity." (note, went through and deleted all old versions Jytdog (talk) 14:20, 22 June 2013 (UTC))

Current proposal as of June 22: Include: "Content discussing the toxicity of substances falls within this guideline. Primary studies should not be used to support content that discusses toxicity." and I will generate a separate essay to Tox for everybody to consider.

Note - original post of this current proposal is at the bottom of this thread; repeated here so it easy to find. Jytdog (talk) 14:20, 22 June 2013 (UTC)

Thanks for considering this.Jytdog (talk) 12:50, 17 June 2013 (UTC)

  • Support - It's not really changing the guideline at all, just specifically mentioning that there is a need for high quality secondary sources in toxicology and primary sources should be avoided. I think this is a beneficial clarification to the page per the reasons you describe above. Lesion (talk) 13:03, 17 June 2013 (UTC)
Consider "non-drug substances" instead of "not-drug substances"Lesion (talk) 13:03, 17 June 2013 (UTC)
argh, typo. I fixed it. Thanks! Jytdog (talk) 13:11, 17 June 2013 (UTC)
  • Oppose per WP:CREEP. It's not clear how this would work with respect to a substance like asbestos, say. There's lots of reputable, secondary material out there and I'm not sure how you'd determine whether it passes this toxicology test or not. Different countries may have different professional standards and bodies covering such matters. Warden (talk) 14:31, 17 June 2013 (UTC)
Colonel Warden, I would love to hear your thoughts on how to manage toxicity content without more explicit language in MEDRS dealing with it.. happy to hear from you on my Talk page or yours, if you like. Or maybe you don't agree that there is a problem (if you have not considered this before, please see the Bisphenol A page and its Talk page to see what I mean) 14:58, 17 June 2013 (UTC)
  • Oppose This is a major change. Jytdog proposes that he be able to reject systematic reviews that are directly about toxicity unless he agrees that the person performing the study should be described as "a toxicologist". The meat of the proposal is "should be sourced from secondary or tertiary sources written by toxicologists". This invites us to produce articles with undue attention to the views of one specialty, to ignore toxicity entirely if no True Toxicologist™ happens to have written a review about it, and to have disputes over whether each author really "counts" as a toxicologist. WhatamIdoing (talk) 14:36, 17 June 2013 (UTC)
Good point. Consider rm "written by toxicologists". Lesion (talk) 14:42, 17 June 2013 (UTC)
Done. thank you both! I am willing to strike "written by toxicologists" as long as the "critical review" language remains. See for example this source that was recently presented (right hand column, about halfway down, paragraph starting "The herbicide glyphosate..." - just cites some primary studies and doesn't comment on their design or relevance - this is the disastrous kind of secondary source that should be excluded) WAID, I am laughing about the TM, but this is not intended just for me (although I want to be able to reach for it) - it is intended for everybody. Hopefully you can accept this with the "written by toxicologists" struck out? Jytdog (talk) 14:58, 17 June 2013 (UTC)
What counts as "critically reviewed"? For example, do all systematic reviews count, or only those systematic reviews that choose primary sources according to some factor like the exposure level? WhatamIdoing (talk) 22:19, 17 June 2013 (UTC)
That is a gorgeous question. While there are advantages in policy writing to being ambiguous, my intention is that the amendment call for the critique of each primary source to be based on toxicological principles - hence, "critically reviewed for their relevance to humans under typical exposures and doses". I would primarily like the amendment (if it gets in) to disallow 2ndary or 3rdary sources that just uncritically cite primary sources; ideally the primary sources would be critiqued under tox principles. Would be happy to discuss changes to accommodate concerns that raises...Jytdog (talk) 22:44, 17 June 2013 (UTC)
follow up, WAID - if what you are looking for is "systematic review" as opposed to "critically reviewed for their relevance to humans under typical exposures and doses" I could live with that as it would exclude what I want to exclude...Jytdog (talk) 22:56, 17 June 2013 (UTC)
I don't really think you'll be happy with that outcome. Consider two papers: a mindlessly performed systematic review of garbage primaries, and a thoughtful literature review (perhaps by a team of respected toxicologists) who tell you that the primaries are all garbage and therefore nobody actually knows whether there are any significant toxicity issues. Which one do you want to be using? WhatamIdoing (talk) 11:10, 20 June 2013 (UTC)
Can a systematic review be "mindlessly performed"?  :) Anyway... based on Colin's feedback I had to move away from "systematic review" ... please see new proposal below... can you help me navigate between you and Colin? Thank you again for commenting. Jytdog (talk) 13:01, 20 June 2013 (UTC)
  • Oppose The entire basis of this proposal rests on Jytog's claim that Wikipedia has a "widespread problem" with content stemming from research that has been "conducted by basic researchers with no training in toxicology." The editor claims to abhor "cognitive bias" yet believes a presumptive statement such as this justifies amending the guideline? Are we now going to willfully discriminate against interdisciplinary science? Semitransgenic talk. 15:39, 17 June 2013 (UTC)
Hi semi, yes I do abhor cognitive bias; that same page notes that I also abhor BS. The way that the results of some experiments are blown up - sometimes by their authors, sometimes by the press, sometime by advocacy groups - to make huge claims about toxicity is sometimes BS. Sometimes attacks on those studies are BS. Sometimes not, all around. The point of the proposed language, is that it takes scientific expertise to interpret and evaluate these studies to make useful, reliable claims about toxicity. I have already accepted the striking of "by toxicologists" so critical reviews by anybody (as long as they review critically for dose/exposure) would be OK. I want to help lift Wikipedia out of the BS-y garbage heap of alarmist claims about in vitro studies and alarmist attacks on those studies - Wikipedia should provide reliable, consensus-driven content about health, not driven by RECENTISM or the most recent controversy about some chemical.Jytdog (talk) 16:13, 17 June 2013 (UTC)
Do you see what I mean, semi? Can you support this amendment, or are there any changes that would make it more acceptable to you? Thank you. Jytdog (talk) 20:10, 17 June 2013 (UTC)
unfortunately i find your additional comments above (I would primarily like the amendment (if it gets in) to disallow 2ndary or 3rdary sources that just uncritically cite primary sources) problematic, this is turning in to an agenda based creepfest. Semitransgenic talk. 23:35, 17 June 2013 (UTC)
Thanks for replying - it is unclear to me why having only systematic reviews count as useful, is problematic to you. But if there is no way we can work to find acceptable language, then I'll just say that I appreciate your time and your responses. Thanks again.Jytdog (talk) 23:50, 17 June 2013 (UTC)
  • Support the revised version (without the demand that a toxicologist review the data). Misinterpretation and panic based on toxicity data can lead to extreme things, such as the recent RCOG report in the UK essentially telling pregnant women not to buy new cars or furniture. Common sense really needs to prevail in this area, and the guideline should be seen to support common sense. JFW | T@lk 15:57, 17 June 2013 (UTC)
  • Support Toxicological info as with most medical content should be based on secondary sources. Doc James (talk · contribs · email) (if I write on your page reply on mine) 16:14, 17 June 2013 (UTC)
  • Support updated wording. I don't see this as a fundamental change, it simply makes more clear what the guideline already says. I've come across enough questions in this area to agree that the guideline should be clarified. Zad68 03:52, 18 June 2013 (UTC)
  • Oppose As the above editor Zad68 mentions, this is not a fundamental change. It doesn't make it more clear. It is quite wordy and just adds more burden to the poor readers trying to get a grip of the sprawling landscape of Wikipedia rules and regulations. II | (t - c) 04:08, 18 June 2013 (UTC)
  • Oppose most of it could be shortened to "Primary studies should not be used to support content that discusses toxicity." What if there is a primary study that says something is toxic, and secondary sources haven't caught up to it yet. It needs to be documented on Wikipedia. I think you are referring to the ethics of primary studies to imply its use for humans. Which I see no harm of ethics there. There is already a section that says preclinical tests results shouldn't be assumed for human implications.
Separately, as for ethics of a study, it should be inserted separately from this. in terms of ethics for animals, primary studies that are in situ and in vitro studies should be encouraged. I would need advice on studies that are in vivo, since the tradeoff for knowledge and for what may be unethical animal tests. Having a guideline that discourages entry from in vivo sources would encourage tests that are in situ or in vitro. I see nothing wrong with including in situ and in vitro studies that are primary. Requiring secondary sources only, does not stop this. - Sidelight12 Talk 04:47, 18 June 2013 (UTC)
Sidelight, if you see nothing wrong with including in vitro studies that are primary, it is no wonder you oppose this edit. Your views do not reflect the MEDRS guideline at all. Wikipedia should not have a mandate to document information in primary sources that the secondary sources have not caught up with. This is what MEDRS is all about. Lesion (talk) 08:49, 19 June 2013 (UTC)
My view is medrs related. See the section: "Avoid over-emphasizing single studies, particularly in vitro or animal studies" it says these sources are invaluable, and allows their use if used with care. medrs is made up of a collection of contributions by different people's thoughts, and it is a living document, so to say that makes no sense. As much bias as there is against primary sources, medrs doesn't prohibit them. I believe there may be a guideline or reason for a guideline to dictate primary sources, but I don't believe they should be dismissed. One is in place already, and I may understand there being something like this in place for toxicology. - Sidelight12 Talk 10:40, 19 June 2013 (UTC)
  • Comment I think the current text (without the "written by toxicologists" part) is in alignment with MEDRS and helpful explanation. But I'm concerned that MEDRS is already too long. There's no reason we can't have a separate (short) page giving advice on toxicology. It would be classed as an Essay rather than a Guideline, but plenty good advice pages on WP are so-classed. This could be linked to from MEDRS. WP:TOXIC anyone? -- Colin°Talk 07:39, 18 June 2013 (UTC)
Thank you for commenting. I would greatly prefer to include this in MEDRS. Would you accept a much shortened version, something like this? Jytdog (talk) 03:59, 19 June 2013 (UTC)

"Content discussing the toxicity of substances falls within this guideline, and should be sourced from secondary or tertiary systematic reviews. Primary studies should not be used to support content that discusses toxicity."

The first clause is fine and may be helpful to include. I oppose requiring systematic reviews. Or any one kind of source like "reviews". The vast majority of issues to toxicity are uncontroversial and can be sourced to any number of reliable documents. We must be careful not to write policy/guideline on the basis of solving only a few specific hard problems because then we raise the bar so high that all the easier articles can't be written at all. Colin°Talk 09:24, 19 June 2013 (UTC)
I hear you Colin... kind of. But I believe you have stated here many times, that we need to ensure that health related content in Wikipedia reflects the actual consensus. With toxicology content, especially with the rising concern about the risks of "chemical exposure" since the 1970's, there is more and more pressure to include content that is scary and does not reflect the consensus, but instead reflects the most alarmist sides of topics where there is no consensus. Jfdwolff mentioned the recent RCOG advisory, which is here: http://www.rcog.org.uk/files/rcog-corp/5.6.13ChemicalExposures.pdf - if you look at that, it warns pregnant women away from an enormous number of things, based on pretty vaguely defined risk. At Wikipedia, we really need a higher standard for tox to avoid this kind of alarmist discussion. It is a problem and it is only getting worse. I am willing to back off systematic reviews, but would like to keep the qualifier, but only for products that are currently marketed. That way, the standard for content on those, is spilt off from tox content about products that have been withdrawn b/c their tox is well established (e.g. PCBs, asbestos). Also am willing to dump the last sentence. Current version with those modifications:

"Content discussing the toxicity of substances falls within this guideline. Content on the toxicity of substances that are still marketed should be sourced from secondary or tertiary sources in which primary sources are critically reviewed for their relevance to humans under typical routes of exposure and doses."

I disagree with dismissing primary sources altogether, but I might agree to there being a certain guideline for their use in toxicology. I think your concern is promotion of, or conflict of interest for substances. Something like primary sources must be peer reviewed, and if there is a reason for conflict of interest for this source consensus can argue against its use. Another guideline to prevent this use may be needed. I don't want that card used it used if this is not the case, just because someone doesn't like it. - Sidelight12 Talk 10:50, 19 June 2013 (UTC)
Using primary sources should be done rarely if ever, for health information. There is almost never a justifiable reason to use one. Jytdog (talk) 02:28, 20 June 2013 (UTC)
  • Comment - The only potentially controversial aspect left here is that the edit specifies systematic reviews, whereas MEDRS for other topics requires only "reliable" secondary and tertiary sources. This wording you have compromised with after an editor above challenged the use of the phrase "critically reviewed". Also what is a tertiary systematic review? A systematic review of other systematic reviews? Not heard that term before. I think your argument is, that merely saying secondary sources is not a tight enough restriction on sources for toxicity, because the primary in vitro studies need to be interpreted by experts in the field to be of any relevance, and secondary sources which just cite these primary sources are not suitable sources? Maybe instead of using the term "systematic reviews" just use "reliable", and if need be qualify this afterwards with a rough description of what reliable means in toxicology? Lesion (talk) 09:07, 19 June 2013 (UTC)
Thanks Lesion. That is a great point, "tertiary systematic review" is goofy. I am getting stuck between Colin and WAID here, which is what I suspected might happen. Gonna try another modification above... Jytdog (talk) 02:28, 20 June 2013 (UTC)
  • Work needed: I agree with your goals, but the proposal is much too wordy, containing (still) quite a few clauses which are redundant and already part of MEDRS. WP:CREEP-- please try to substantially reword so that the new text doesn't repeat what the guideline already says. Of course toxicology is part of MEDRS. Of course we should rely primarily on secondary sources. We don't need to repeat all of this. I support the proposal intent if the wording is shortened; I oppose the wording as written. SandyGeorgia (Talk) 03:06, 20 June 2013 (UTC)
    • It might be useful to write up an essay talking about how to evaluate a tox paper. It might be useful to editors who are trying to do the right thing (i.e., those not trying to win a dispute) as well as helping crystallize a few points that would be useful here. WhatamIdoing (talk) 11:10, 20 June 2013 (UTC)
  • Oppose I'm concerned about the consensus which seems to be developing here, which I believe threatens to become a classic case of WP:LOCALCONSENSUS. I know it's only on the talk page, but when an editor says that "at Wikipedia, we really need a higher standard for tox to avoid this kind of alarmist discussion" and the "alarmist discussion" is a paper from the Royal College of Obstetricians and Gynaecologists, something is wrong. Wikipedia should, of course, summarize published material from such a source in an entirely neutral way regardless of what editors think about it. The purpose of MEDRS is in danger of being lost in Wikilawyering about primary/secondary/review sources. The purpose is to prevent Wikipedia being used to peddle quack remedies and make claims which may damage the health of readers if they act on them. Claims of medical efficacy and warnings of possible toxicity are not the same; much higher standards of evidence should be required for the first than the second, where the precautionary principle applies – although in all cases the strength of the evidence should be reflected in the writing. If a herbal product meets the standards of notability through its widespread use, as documented in reliable sources, and there are reviews which meet the standards of MEDRS to show some degree of efficacy plus some recent primary reports of toxicity not yet reviewed, it would be irresponsible censorship not to mention (in a properly qualified way) the possible toxicity in the article. (It might even expose Wikipedia to legal action in some jurisdictions should the toxicity be confirmed and the product cause harm to someone and it could be shown that Wikipedia editors knew of the reports of toxicity and suppressed them.) Peter coxhead (talk) 09:21, 20 June 2013 (UTC)
Peter, I strongly disagree that we should worry about "legal action". Please read the terms of use, disclaimers etc. before you make such suggestions. Irresponsible? - possibly this could be argued, fine. Illegal? - this is suggestion is a joke and does not qualify as legitimate point in this argument. No-one is going to sue me, or you any other Wikipedia editor (anonymous volunteers) if they fail to add a scientific paper about the toxicity of a substance to Wikipedia, with these terms of use etc in place. Lesion (talk) 09:33, 20 June 2013 (UTC)
Legal action or not, that is a valid ethical reason. If knowledge is available I dislike discriminating against it, when nothing is wrong with it. A vast amount of information is in primary sources that aren't in secondary sources. With good reason some primary sources may be invalid to the context of an article. What about a guideline that says sources must be peer reviewed or reliably published. Also, the source must be third party to affiliations of a substance. Third-party sources can be primary, secondary, or tertiary. - Sidelight12 Talk 11:17, 20 June 2013 (UTC)
Are you talking about changing the wording of this proposed edit about toxicology sources, or about changing MEDRS generally? If latter, new section with specific proposed changes delineated please. Lesion (talk) 11:57, 20 June 2013 (UTC)
It covers toxicology, but for instances I can't see now, it could cover more than that. but ok, I might propose this soon to deal with this issue. One discussion at a time seems proper. - Sidelight12 Talk 12:09, 20 June 2013 (UTC)
Sidelight above you write " If knowledge is available" - and this is exactly the point. MEDRS exists because we only want to state what is known (or what the medical community believes is true as stated in 2ndary and tertiary sources, which is the closest we can come) - and there is too much content in WIkipedia making claims that X or Y is toxic in A or B ways based on sources that don't take toxicology into account - heck there is too much such content based on primary studies. In o other words, too much content stated something as "known" when it is not known.Jytdog (talk) 12:58, 20 June 2013 (UTC)
Ok, forget the legal aspect (although in many countries attempts to disclaim responsibility through things like terms of use have no legal force). Please answer my main point that efficacy and toxicity aren't the same. The latter is subject to the precautionary principle. We shouldn't state on the basis of primary studies that a particular treatment is efficacious. We should state on the basis of reliable primary studies (preferably plural here) that there are preliminary reports of toxicity. Peter coxhead (talk) 19:15, 20 June 2013 (UTC)

I disagree strongly with you Peter. It is exactly because of the philosophy you are stating here that I wish to strengthen MEDRS specifically with respect to toxcity. MEDRS states "Therefore, it is vital that the biomedical information in all types of articles be based on reliable, third-party, published sources and accurately reflect current medical knowledge." This sentence (and the guideline) makes it clear Wikipedia should not include content that is "preliminary" about any health related matters, and especially not about emotionally laden preliminary conclusions that "X might be killing your baby". The goal is not to be "cutting edge" but instead to be the fat old cow walking at the end of the wagon train - we provide content on what is known - not what is speculated or preliminary. Wikipedia is not the place to go for the latest medical news. It is a place to find reliable knowledge as reflected in secondary sources. Jytdog (talk) 19:45, 20 June 2013 (UTC)

Strongly agree with Jytdog, and s/he has said it better than I could. MEDRS is not just about efficacy of treatments, but about any matter pertaining to human health. People going on about using primary sources here need to actually read MEDRS and WP:RS and realize there is encyclopedia wide policy relating to not using primary sources. Lesion (talk) 20:00, 20 June 2013 (UTC)
I have read (and regularly use) MEDRS, and I'm also very familiar with WP:RS. But editors here need to stop Wikipedia:Wikilawyering by endlessly referring to WP guideline documents, and deal with the issues: "Wikipedia policies and procedures should be interpreted with common sense to achieve the purpose of the policy..." Discuss the issues. Peter coxhead (talk) 20:20, 20 June 2013 (UTC)
Hi Peter, I tried to discuss both the letter and the spirit of policy and guideline, specifically with respect to toxicity. And I tried to directly speak to what you said about warning people. Please tell me what part of the issue you feel I have not addressed. Thanks! Jytdog (talk) 20:24, 20 June 2013 (UTC)
My comment was directed at the one above, by User:Lesion, rather than yours. But the point remains that it's not helpful to keep re-referring to MEDRS when we are discussing what should be in MEDRS. The discussion needs to focus on the purpose of the policy. So I look forward to your reply to my question below. Peter coxhead (talk) 20:34, 20 June 2013 (UTC)
  • Weak support. While I agree with the implication of the current revision, it seems to be unnecessarily long-winded. How about something like "Information about the toxicity of substances (both drugs and non-drugs) should be supported by secondary sources." Axl ¤ [Talk] 10:52, 20 June 2013 (UTC)
Thank you for the support, weak as it may be! The wordiness is unfortunate - I have been trying to carve this down to the minimum that (I think) is necessary. The pressure that some editors feel to provide "fair warning" to readers about possible toxicity of currently marketed products (e.g comments by Sidelight and Peter Coxhead above) drives them to want to generate content based on conclusions drawn from primary sources (already pushing the envelope of both PSTS and MEDRS); for them (I believe), tox content is different from efficacy content and there should actually be a lower bar for adding content -- and this is exactly what has been concerning me. I don't think it is enough to just say "content about toxicity falls within MEDRS" - I think we need, within the higher standard that is MEDRS, a slightly higher standard again for content about toxicity for currently marketed products. And that takes more words. I hope that makes sense and that this was not too wordy. Jytdog (talk) 20:02, 20 June 2013 (UTC)
What is the rationale for requiring a higher standard for toxicity than efficacy? Please explain. Peter coxhead (talk) 20:20, 20 June 2013 (UTC)
Thanks for your note above. Happy to answer. I understood you to say above, that you approach tox content differently from efficacy content - that due to the precautionary principle you think that it is Wikipedia's responsibility to warn people about possible toxicities of currently marketed products based on preliminary findings described in primary sources. That is the first time I have heard the idea articulated so clearly (which I appreciate!), but in many Talk pages I have encountered similar motivation, not so clearly articulated. This approach is well-intentioned but the intention violates MEDRS and NOT/RECENTISM, and enacting it violates PSTS and MEDRS in the reliance on primary sources. I made my proposal to prevent this well-intentioned but misguided approach to tox content about currently marketed products; the higher standard serves as a counter-weight against the moral urgency some editors feel to (mis)use Wikipedia to warn the public about these possible dangers. I don't know if you agree but I hope you see the reasoning. Jytdog (talk) 20:53, 20 June 2013 (UTC)
Just re-read this and I think it is kind of dickish. Sorry about that. My intention wasn't to be so harsh, but it was to be clear. Hopefully it is clear. oy. Jytdog (talk) 21:28, 20 June 2013 (UTC)
Don't worry, I wasn't offended. Your argument for requiring higher standards for toxicity is basically to forestall what editors who disagree with you might do. Firstly, this is a poor reason: the reason should be based on a rational appraisal of the issues, not the hypothetical actions of other editors. Secondly, it's pointless, since those who disagree with you in this respect either haven't read MEDRS or aren't going to be influenced by the requirement for the same standard, let alone a higher standard.
A key issue for me is the requirement for editors to reach decisions by consensus on specific content in specific articles taking into account WP's guidelines. Trying to produce over-rigid guidelines to cover all cases (including hypothetical ones) is counter-productive (as we've seen in recent actions at the MOS generally).
My concern is not primarily with currently marketed "orthodox" medical products, and Wikipedia must never be responsible for the kind of scare which stopped parents in the UK giving their children the MMR vacine based on one individual's flawed "research". But many articles on plants which have been used in traditional herbal medicine do pose problems. Where reliable secondary sources attest the traditional and/or ethnobotanical uses of the plant, this information meets Wikipedia's requirements for notability and verifiability – verifiability in terms of the plant's cultural history and use, of course, not in terms of its efficacy. We can – and I regularly do – remove attempts to claim efficacy not supported to the standards of MEDRS. But for many readers, the mere fact that the plant has been used as a "herbal remedy" implies, even though we carefully do not say this, that it may be efficacious. Reviews of quality studies of the toxicity of many such herbal products do not exist; you surely know as well as I do that what gets reported in journals is what scientists get funded to research. For example, for plants used in ayurvedic "medicine" there are large numbers of papers and reviews in Indian journals, but very few reports of toxicity issues, even though these are known to be a problem. It cannot be acceptable to be able to say that X is used in ayurveda, and is believed by ayurvedic practitioners to be effective in the treatment of Y, but not be able to say that a primary study by Z suggests caution in the use of X as toxicity has been reported.
In summary, if it were possible to completely remove the many, many descriptions of traditional herbal "remedies" in Wikipedia, and insist that no report of traditional use can be made without evidence of efficacy or non-efficacy to MEDRS standards, then I might agree with you. But it isn't possible. So I don't agree with you. Peter coxhead (talk) 14:24, 21 June 2013 (UTC)

Hi thanks for not being offended, and for the discussion. It is a little unfair to suggest that I want to amend the guideline "basically to forestall what editors who disagree with you might do." The desire to amend definitely comes from several actual difficult conversations; what that revealed to me was that the guideline was not providing guidance sufficient to keep alarmist, and toxicologically poorly done, material about toxicity out of wikipedia. I imagine this is the same impulse that produced this guideline in the first place -- a desire to have something to help resolve disputes about what is proper to include in Wikipedia. I hear you - a lot - on the issue with traditional medicines. I have done work on articles like that too. It is a struggle to create and keep MEDRS content on them in place. However, while I share your concern, it is off topic for this request for amendment... Jytdog (talk) 22:59, 21 June 2013 (UTC)

If it were agreed that higher standards were required for toxicology reports than efficacy reports, this would (of course) also apply to articles concerned with traditional medicines. Since there are many such articles, it would potentially have a significant numerical impact. So it's not "off topic", it's very much "on topic". The way to keep alarmist, poor material out of Wikipedia is to deal with it article by article. If it's truly alarmist and methodologically poor, it can be removed for that reason, and doesn't need changes to MEDRS which may have unforeseen consequences ("you can write that herbal product X is efficacious because the sources just meet the MEDRS standards but not that it might be toxic because this needs a higher standard"). Peter coxhead (talk) 08:24, 22 June 2013 (UTC)
NOW I understand, thank you for explaining. My amendment was intended for "non-drug substances" (which has now fallen out of the amendment)... and I see what you mean that even if the "non-drug substances" were back in, traditional medicines (which are essentially dietary supplements) are "non-drug substances" too! Had not considered that and it is a great point. Especially with the language as it currently stands. Colin's suggestions about a separate essay is starting to look more sensible, to deal with all this.

Current proposal: Include: "Content discussing the toxicity of substances falls within this guideline. Primary studies should not be used to support content that discusses toxicity." and I will generate a separate essay to Tox for everybody to consider.

Note - I am going back through and deleting previous proposals so this is easier to track. Jytdog (talk) 14:20, 22 June 2013 (UTC)
I think it would be desirable for you to start with the essay. Perhaps you could use diverse classes of examples, like pesticides, prescription drugs (vaccines?), herbal treatments, and vitamins, to explain the typical concerns that editors have. Also, I hope that it will emphasize the one important area of agreement, which is that if excellent secondary sources exist, then they ought to be relied upon. WhatamIdoing (talk) 14:33, 27 June 2013 (UTC)
  • Toxicity is already covered by MEDRS, and much of the material is redundant since it is already in the guideline. Maybe have a short sentence listing some of the topics covered by MEDRS instead? IRWolfie- (talk) 09:54, 22 June 2013 (UTC)
  • Colin's suggestion makes eminent sense. The use of "non-drug substances" phrasing is potentially problematic in the context of CAM, at least in the US. Most herbals are normally considered as food, but garlic "for hypertension" or even "relaxing" chamomile must properly be considered a drug. Almost anything found in homeopathic materia medica is inert as used (or at most has a tiny dose of ethanol as dilutant). These things still must be considered as drugs because of associated indications, irrespective of whether there is any sign of either specific efficacy or toxicity. As Mr Bumble says, "The law is a ass", but it still is the law. Until there's an outbreak of sanity among legislators we're stuck with it, so let's ignore the drug/non-drug distinction and focus on the toxicity question. How about wp:TOXRS, aka Wikipedia:Identifying reliable sources (toxicity) as a working title for the essay? I'm thinking that we'd generally treat things such as ClinicalTrials.gov or statistical summaries from poison control centres as (low grade) secondary sources, just barely good enough to use with cautious wording. I'd expect some tension at play between avoiding alarmism on one hand and applying the precautionary principle which is so dear to environmentalists on the other. LeadSongDog come howl! 15:18, 27 June 2013 (UTC)
  • The problem you appear to be having is reviews you don't like rather than editors citing primary sources directly. Also, people aren't going to change their votes based on the revised proposal, so this is dead unless you start a new section. As a general comment, as WhatamIdoing has articulated quite clearly above, I don't think you'll have much luck accomplishing what you want through editing policy pages. Personally, I don't think policies should be edited to fit specific disputes (see #8 in the cynic's guide). Your introduction and later comments revealed that you believe "we need, within the higher standard that is MEDRS, a slightly higher standard again for content about toxicity for currently marketed products". Your proposal included restrictions upon using reviews based upon the author (a "true" toxicologist), the type of review ("systematic"?) and how much analysis the review gives to the sources it cites. These look very susceptible to gaming. To get into the weeds of one example you cited (Bisphenol A), you're probably not going to be able to remove the perspective of the Chapel Hill scientists such as vom Saal just because the reviews discuss a lot of animal studies and regulators from the FDA (altho not necessarily their scientific advisors) and EU don't really agree. II | (t - c) 08:45, 29 June 2013 (UTC)
Hi ImperfectlyInformed - you are pretty much missing my point. Chapel Hill is not part of my problem with the BPA article and other articles. The main problem is the train wreck of primary sources which under PSTS and MEDRS should not be there at all. There are also some really terrible 2ndary sources cited for tox content that don't review the primary sources they cite using basic tox principles (I linked to some above relevant to BPA and glyphosate), but I have already given up the idea that I can change this guideline to exclude them, as you can clearly see in the current proposal - I have accepted that making that change would affect too many established articles. My issue with some reviews is not that I "don't like" them - by which I think you mean their conclusions - but rather that some secondary sources that make tox claims, fail to use toxicological standards and methods when citing primary sources, making their claims unreliable. The issue is quality of the review through which conclusions are arrived at, not the content of the conclusions themselves. Reliable toxicology says what it says. Which is sometimes grey, not black and white, which is fine, as long as it is actual toxicology and not just hand waving. As to your broader comments - as I have written here a bunch of times; toxicology is a discipline (an important one) within the biomedical sciences and there are way too many "armchair toxicologists" out there. I already have adapted the proposal to the several criticisms so I am not sure why you are rehashing old stuff. In any case, am looking to finish the draft essay this weekend and hope to post it Sunday evening. Will be fun. Jytdog (talk) 09:24, 29 June 2013 (UTC)
The topic of low-quality reviews has come up before - see for example a discussion from 2008 called Dated and low-quality reviews. Back then I was (not very coherently) raising the question of low-quality reviews, and I was also concerned about obscuring primary sources behind the uncritical parroting of a narrative review. There's no simple solution to that problem. Reviewing Bisphenol A, it looks to me like your approach was more deletion rather rewriting, and you also lacked persistence. You have to keep hammering away and incorporating suggestions to produce a consensus. Ultimately I think most people would be happy to see that particular article trimmed down to summary style. By the way, as you probably know there are plenty of scientists who think the Chapel Hill "consensus" is alarmist, so it's a little hard for me to understand where you would draw the line. I mean, if you've got the FDA and EU regulators disagreeing with a source, how is that source OK but the source which you personally (but perhaps no sources) disagree with as "fail[ing] to use toxicological standards and methods" not OK? (I'll admit the disagreeing sides here make it complicated.) This gets into the thorny issue of editors making authoritative judgments on which reviews are valid. I am of course not opposed to editorial research, and I've been involved in cases where, based on editorial consensus, reviews were not used because they were low-quality. However, rewriting the guideline ultimately doesn't change anything: you have no way to prove that you're right, so it comes down to editorial consensus. II | (t - c) 10:09, 29 June 2013 (UTC)
Interesting, nice to see that you struggled with this too. MEDRS itself puts boundaries on the kinds of sources that can be used, and in the Assess section, calls for editors to look at the source and consider what it is before using it. For interventions, meta reviews of RCTs are best, "Systematic reviews of bodies of literature of overall good quality and consistency" are next, etc etc. I was looking to add something to MEDRS along those lines for tox, where it would be, for tox, "reviews of bodies of literature of overall good quality and consistency that take into account route of exposure and typical doses to which humans are exposed". It is not a big leap. Your line of critique about what I "personally" view as as X or Y when no other source calls it that, a) runs up against MEDRS's call for editors to assess, and b) is a red herring - as you well know it is very rare for one review to critique another directly. And again I am not out to drive content in one direction or another. I am trying to get rid of tox content based on sources that don't address basic tox principles. Toxicologists can disagree with one another; it is not for me or Wikipedia to sort that out but rather just to describe the mainstream viewpoints. On endocrine disruptors, my understanding is that mainstream toxicology has consensus that it is a good thing that DES was taken off the market and that there is a risk that other chemicals on the market (and chemicals endogenous to various foods) ~may~ be doing similar harmful things in a more subtle way, but I think that is as far as the broad consensus goes. From there you go to individual chemicals and you find ranges of tox opinion about various risks of harm. The science appears to me, to be somewhat at the end of its rope right now, until more, and carefully done, research is completed. I am very heartened that there is a concerted effort going to coordinate research among academics, the NIH, and regulators to get definitive answers to questions raised about BPA by the Chapel Hill Group, the big NTP review, and the decisions/report by the FDA based on both reports http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3548284/ see especially this ("The consortium represents an unprecedented approach to conducting GLP-compliant research by bringing researchers and regulators together during the planning stage to ensure that results will be maximally useful for risk assessment and regulatory decision making. The grantees and FDA representatives, along with coordinators from the NIEHS/NTP, held their first in-person meeting in March 2012. This collaboration is expected to produce a robust and valuable body of work on the effects of BPA in rats, a key animal model in toxicity testing.") So no, I don't think the Chapel Hill report is outside the consensus and I have never stated a desire to remove it. Jytdog (talk) 15:21, 29 June 2013 (UTC)

DARE guidance?

I'm wondering whether the DARE Database of Abstracts of Reviews of Effects [10][11] could somehow fit into MEDRS as a useful tool to gain unbiased guidance on the reliability of a given systematic review. For many recent reviews on their database [12] DARE provides brief critical appraisals (box here [13]) of the source, with nuanced guidance on the reliability of the authors' conclusions (also accessible via TRIP). For example:

  • Questions about the quality of included data and methods of synthesis mean that the authors' conclusions should be treated with caution. [14]
  • In general, the authors’ conclusions appear to be appropriate, although they depended on the model assumptions. [15]

As an individual contributor, I find guidance such as this useful for nuancing the weight to give to the findings of particular systematic reviews. 86.161.251.139 (talk) 10:13, 11 July 2013 (UTC)

Agree - less than half (and reducing) have those appraisals, but they're great when they're there. There are several things to know about it - not least of which is to be careful if not going to DARE itself, how often it is updated where you're using it, because there are big differences, and how to request an appraisal. I'm building up a file with quite a few things like this missing from the systematic reviews page.Hildabast (talk) 02:16, 12 July 2013 (UTC)

Reverted edit

I have concerns about a statement in the "Biomedical Journals" section:

  • It is usually best to use reviews and meta-analyses where possible, as these give a balanced and general perspective of a topic—and are usually easier to understand.

While agreeing that that narrative reviews tend to be relatively reader friendly, the same can scarcely be said of meta-analyses. For example, here is a recent open-access report I randomly selected on PubMed: [16] [17]. I doubt many general users of Wikipedia would find that easy reading. Also, I don't think that meta-analyses (a particular type of systematic review) can be said to provide a "general perspective of a topic". For methodological reasons, they have to address a rather specific research question. As noted elsewhere in the section, "whereas a narrative review may give a panorama of current knowledge on a particular topic, a systematic review tends to have a narrower focus."

I don't see the sentence adds anything useful to the paragraph and I suggest removing it. 86.161.251.139 (talk) 15:06, 11 July 2013 (UTC)

Disagree... the particular meta-analysis you selected would be readable to an editor familiar with the work in the field, I'd think. And it's a particularly "deep" example, in my opinion. Here's one that I think is more typical: [18], conclusion is "On the basis of the current research, any clinical benefit of oral glucosamine for patients with chronic LBP and radiographic changes of spinal OA can neither be demonstrated nor excluded based on insufficient data and the low quality of existing studies." - that's very accessible language to a typical editor. Don't forget that we're talking about recommended sources for editors to use. The job of the editor is to take the source and represent it accurately and accessibly to the end-reader. I find meta-analyses are useful, they're usually a dozen pages of methodology followed by one paragraph of conclusion, and it's just that conclusion we're looking for, as editors. Zad68 15:28, 11 July 2013 (UTC)
Maybe you've misunderstood my point... I've absolutely nothing against encouraging contributors to use meta-analyses. Far from it! And I fully take your point that meta-analysis conclusions have to be clearly defined. However, saying that meta-analyses, with their pages full of methodological technicalities, are "easier [than what?] to understand", seems—as written—a pretty strange claim to make. Since any meta-analysis worth its salt addresses a sharply defined PICO research question, it's hardly the ideal port of call for a "general perspective of a topic"; as pointed out a few lines below, that's the role of a good narrative review.
Fwiw, the paper you cite as being more representative [19] is a systematic review which does not include any statistical meta-analysis. Here's a randomly generated sample of five more free-access reports of clinical meta-analyses recently indexed by the NLM: PMID 23639488, PMID 23637132, PMID 23587198, PMID 23558282, PMID 23558164. 86.161.251.139 (talk) 17:41, 11 July 2013 (UTC)
Ok so if the issue isn't that meta-analyses should not be recommended, but that it's incorrect to call meta-analyses "easier to understand", we can get somewhere with this. The edit that was reverted made this change: "It is usually best to use reviews and meta-analyses where possible, as these give a balanced and general perspective of a topic—and are usually easier to understand." I think the issue was that the edit removed the characterization of reviews as "easier to understand" and it looks like we all agree reviews are indeed easier to understand (although the guideline doesn't say what is it that they're easier than). I agree that an individual meta-analysis does not often provide a general, easy-to-understand overview of a topic, and they're not really designed to. They may have a paragraph backgrounder and then jump right into the data. Suggested change is:

It is usually best to use reviews and meta-analyses where possible, as these . Reviews in particular give a balanced and general perspective of a topic—and are usually easier to understand.

Comments? Support/oppose? Zad68 17:59, 11 July 2013 (UTC)
No need I can see for an RFC-type discussion with Support/oppose etc... Personally, I think your suggestion is an improvement. Though I would again question whether systematic reviews give a "general perspective of a topic" (they also have to address tightly defined PICO questions), or indeed whether they are necessarily "easier to understand" (they too are tied to strict reporting of their methodology).
In general, I feel that this part of the page could be better explained. (For instance, it would make sense to me to start a section on 'Biomedical journals' with "Peer reviewed medical journals are a natural choice as a source for up-to-date medical information in Wikipedia articles" rather than with "As mentioned above...".) Aside:I'd like to say that I'm a strong supporter of the guiding principles of MEDRS and my reverted edit was only intended as a small correction. I did the edit after making a somewhat more substantial proposal, #DARE guidance?, which I really do feel may be worthy of some consideration (eg to avoid WP:GAME?). 86.161.251.139 (talk) 18:33, 11 July 2013 (UTC)
86.161.251.139 you really seem to know your stuff and I hope you stick around and keep contributing to WP:MEDICINE-scope things. I see your IP User Talk page and won't bother with the "Welcome to Wikipedia, consider registering" song and dance. I feel like I might have run into you already. Be bold, make needed updates. The worst that happens is it gets reverted and we discuss it a bit. I'll put the small tweak to your change into MEDRS and see if anybody reverts it. Feel free to make more suggestions and changes. Zad68 18:41, 11 July 2013 (UTC)

Arbitrary break: "Review" terminology

What wording improvement can we make to make it clear we are saying reviews (PubMed Publication Type = "Review") and not systematic reviews? Zad68 18:49, 11 July 2013 (UTC)

"Literature reviews". But I believe that the PubMed search you name gives both types of review articles: literature reviews and systematic ones. WhatamIdoing (talk) 23:50, 11 July 2013 (UTC)
Regarding the terminology, the distinction generally drawn I think—though Hildabast may correct me—is between "narrative reviews" and "systematic reviews" (both of which are, strictly speaking, "reviews of the literature"). Regarding our wikilinks, I guess the narrative reviews redirect makes the distinction clear through the dab header. 86.161.251.139 (talk) 09:33, 12 July 2013 (UTC)
Well, you're right, but there's a bit more to it ;). That is the distinction usually drawn - but narrative is also used for non-quantitative (so if I can't meta-analyse, I present results narratively). It's not a satisfactory way to describe non-systematic reviews. It's on my mental list to start working on the relevant WP pages, because there's a lot that needs to be done there - having that more complete/accurate may help people decide what changes to make here. One of the biggest problems here at the moment is the guidance on searching: what's described here isn't the best way to search for reviews, because relying on publication types restricts you to MEDLINE-indexed reviews. That's problematic for two reasons: MEDLINE-indexing only indexes MEDLINE journals. Consequently, articles in purely PMC journals and Bookshelf/PubMed Health are missing. And secondly, MEDLINE-indexing takes months - as of a few days ago, for example, the most recent Cochrane reviews MEDLINE-indexed were from February. Combine those two problems, and for a search term like "asthma", you could be missing up to a 1000 reviews, the vast majority of them recent and/or full text. I'm working on an additional option at the moment, and will let you know when we get it organised. For systematic reviews, the option is PubMed Health. We include DARE, Cochrane, so it's non-MEDLINE/PubMed, and we're the ones gradually pulling in all the health technology assessment systematic reviews into the system (that are neither in DARE nor MEDLINE). PubMed Health updates DARE every week (some other places only do it once a month or once a quarter), and we'll be doing it more frequently than weekly sometime soon. Right now, your best bet is to search PubMed Health for systematic reviews, because everything is a systematic review (and there is no publication type systematic review anyway). Currently, when I look for reviews in PubMed, so that I also have a chance of seeing more non-MEDLINE-indexed ones, I do this: AND (review [pt] OR review [tw]).Hildabast (talk) 12:04, 12 July 2013 (UTC)
Sorry - didn't explain what I mean by health technology ones: I mean systematic reviews done by health technology assessment agencies. We digitise these and put them into the PubMed system as well as PubMed Health. Included are things like the NICE clinical guidelines that include full text complete systematic reviews, systematic reviews from agencies like AHRQ, the VA, IQWiG. Some of these also publish articles in MEDLINE journals, but most don't (even though they are peer-reviewed). So they meet our quality review, but not via the MEDLINE process as they are not journals.Hildabast (talk) 12:10, 12 July 2013 (UTC)
Great stuff! To try to keep the thread navigable, can I try to separate out two points here, and respond to them separately, postponing the searching question regarding the #Searching for sources section and PubMed Health etc until after another "Arbitrary break"?
  • Regarding the #Biomedical journals subsection: Does this edit [20] help at all? At this point on the page, it seems to me, we're merely providing all our general users with a very brief (though hopefully terminologically correct!) introduction to the existence of different types of review articles, which may be deemed appropriate as sources in different editorial contexts. Can you perhaps improve the wording while keeping it simple for the benefit of everyone?
    • 86.161.251.139 (talk) 15:25, 12 July 2013 (UTC)

Arbitrary break

While Zad68 was making a detailed edit, I was trying to make a more general (though perhaps hasty) one, along the lines I was trying to suggest above. For now, I've posted it as a bold edit [21] for BRD, if necessary. It will probably need some tidying anyway! 86.161.251.139 (talk) 19:21, 11 July 2013 (UTC)

I agree with the anonymous IP editor. I am happy with the recommended changes. Axl ¤ [Talk] 23:30, 11 July 2013 (UTC)
A few points - MEDLINE publication type review isn't just systematic and non-systematic reviews of literature, and it doesn't include meta-analyses if they are not reviews. I don't agree with the statement that non-systematic reviews are broader: or because of their systematic methodology that they lack contextual information. Go to a literally 1,000 AHRQ review, that includes key questions on most aspects of care, or one like lactose intolerance. Being systematic doesn't preclude these things - and being non-systematic doesn't ensure their presence. There are publication type reviews that are only letters, for example (and quite a few!).Hildabast (talk) 02:28, 12 July 2013 (UTC)
Aren't all (published) meta-analyses also reviews? Axl ¤ [Talk] 10:03, 12 July 2013 (UTC)
A meta-analysis of, say, two major trials may not involve any review (systematic or otherwise). 86.161.251.139 (talk) 10:20, 12 July 2013 (UTC)
I think I understand where Hildabast is coming from. And I agree it's complex...
A premise: As I understand it at least, unlike professional reporting guidelines [22], MEDRS has to serve in practice as both 1) an editorial tool to help ensure unbiased sourcing of medical content in keeping with Wikipedia's principles and culture, and 2) a user-friendly guide to best medical sourcing practice for the entire range of good-faith contributors. Not an easy combination, especially given the key importance of ensuring that the guideline does not lay itself open to abuse. And all this while keeping it as short and sweet as possible... What I think we're cautiously trying to do together here is to enhance the guideline without breaking something that already works pretty well.

In reply to at least a couple of the points Hildabast raises:

  • "I don't agree with the statement that non-systematic reviews are broader." I've now tweaked the wording slightly [23]. IMO, we don't want to discourage in any way appropriate sourcing of up-to-date general encyclopedic information—as distinct from identification/sourcing of specific current best evidence—from recent high-quality, open-access narrative reviews and authoritative textbooks (e.g. Robbins, Harrison's et al). While this distinction may ultimately be a fuzzy one, I think there's general consensus on the project to encourage a judicious mix of high-quality journal and book sources, including open-access narrative reviews which general users can readily reference for more discursive information on a topic.
  • AHRQ reviews. I wholeheartedly agree about the usefulness of AHRQ, NICE and other reliable bodies that systematically evaluate best evidence, and I'd consider this type of source to be our gold-standard "ideal source" for clinical evidence, when available and applicable. Maybe this could be expressed a bit more explicitly? (I also have a cautious idea that the focus of the #Assess evidence quality section could perhaps be enhanced by somehow incorporating AHRRQ, NICE, DARE, etc.)
    • 86.161.251.139 (talk) 12:48, 12 July 2013 (UTC)

I'm parking this at the bottom of this discussion for lack of a better place, in case new editors here haven't encountered it: Wikipedia:Wikipedia Signpost/2008-06-30/Dispatches. Tim Vickers and Eubulides wrote it for the Dispatches in 2008 ... some of it might be outdated (haven't checked), but I think a lot of it is still useful. SandyGeorgia (Talk) 01:30, 19 July 2013 (UTC)

Guidance on searching

I seem to understand that Hildabast is suggesting (above [24]) that PubMed Health has the potential to become the default MEDRS tool for identifying "ideal sources" for clinical evidence. To me that sounds a rather attractive prospect. At present, we provide contributors with only the barest of guidance on #Searching for sources. Could it perhaps also provide an effective filtering tool to #Assess evidence quality?
86.161.251.139 (talk) 16:15, 12 July 2013 (UTC)

Well, I think it's already the main way to look for systematic reviews, for a few reasons - but not for every other type of review, or for recent non-Cochrane systematic reviews (because DARE can take weeks or months). Working on improving options at PubMed now, that has the best of both worlds. PubMed Health has additional content though - PubMed is a database of citations, PubMed Health includes a growing body of evidence-based resources for clinicians and patients. So the situation's in transition. I'm finding watching and participating in WP:Med very helpful actually for considering what improvements would be valuable (including ways of facilitating citation directly - eg a PMHID citation method that incorporated the PMID and the extras at PubMed Health, eg for Cochrane the plain language summary, for reviews in DARE the quality appraisal and so on). Hildabast (talk) 23:05, 12 July 2013 (UTC)
Handouts?
86.161.251.139 (talk) 09:54, 14 July 2013 (UTC)

I suppose I should have read further before parking Wikipedia:Wikipedia Signpost/2008-06-30/Dispatches in the section above; some of it may be useful here. SandyGeorgia (Talk) 01:31, 19 July 2013 (UTC)

There's a confusion here between meta-analyses and systematic reviews. Meta-analysis is a statistical tool for combining the results of studies. They don't need to be trials, and they don't need to be part of a systematic review. It's just a statistical technique. Meta-analyses can be conducted on subsets of studies (eg a pharmaceutical company may meta-analyze its own trials on one of their products). Many systematic reviews have no meta-analyses in them (because there weren't multiple studies similar enough to combine). One has to be really careful with these terms, and not use them interchangeably. One is a type of research project/methodology, one is a statistical technique. Hildabast (talk) 15:49, 24 July 2013 (UTC)
As in the second paragraph of #Assess_evidence_quality? Some tlc needed here, imo. 86.161.251.139 (talk) 15:34, 29 July 2013 (UTC)

Authoritative textbooks

On a related note, it would be nice to get a list of "authoritative textbooks" for each specialty area. People ask for suggestions every now and again, and it might help. I think that the pages for each WP:MEDTF task force would be the place to put it, rather than here. WhatamIdoing (talk) 23:42, 12 July 2013 (UTC)

I was going to add one to the Wikipedia:WikiProject Medicine/Neurology task force but wasn't sure where it would go. And looking at the talk page/history, is there anybody active there to read it? -- Colin°Talk 07:22, 19 July 2013 (UTC)

MEDRS recommmended as a useful guide on how to find info on the web

I have just seen in a blog on medicine and skepticism (science-based medicine) that a comment in a post on how to find medical reliable info on the web cited MEDRS as a good guide.

The link here, the author of the post (not of the comment recommending MEDRS) is Steven Novella.

Most probably the author of the comment recommending MEDRS as a great guide on how to find good sources for medical content on the web is a regular here, but nevertheless I believe it is interesting to know that not only articles are "citable" and useful for people outside the project, but also guidelines, policies, etc.

--Garrondo (talk) 06:24, 19 July 2013 (UTC)

Proposed addition for identifying possible poor journals

While this guideline does a good job of identifying good journals and how to find them in the last paragraph of the Biomedical Journals section, I do not believe it does a good job of identifying poor journals (how to spot one, what to do when you have one). I bring up this proposal now because as I have noticed a large uptick in what can only be described as bad journal articles when I do a standard PubMed search. Unfortunately, PubMed Central is now indexing a number of non-MEDLINE indexed journals of dubious quality due these journals meeting their open access criteria (somewhat related previous discussions about open access journals and their reliability can be found here and here). Given the recent publicity about predatory journals, this problem will likely get worse, so I think we should deal with this now. This proposal is not meant to discuss open access journals per se, as there are plenty of good open access journals and poor print journals. However, I think we need to do a better job of directing editors away from poor journals, as well as dealing with the POV pusher who insists on using them.

Proposed addition:

An integral part of finding high quality sources is avoiding articles from journals without a reputation for fact-checking and accuracy. A red flag that a journal article is probably not reliable for health claims includes publication by a publisher that has a reputation for exhibiting "predatory" behavior, such as Bentham Science Publishers, Dove Medical Press, or OMICS Publishing Group, among others. These publishers may have questionable business practices and/or peer-review processes that raises concerns about the reliability of their journal articles.[1][2] Other indications that a journal article is probably not reliable is its publication in a non-MEDLINE indexed journal, or the publication of an article whose content is outside the normal scope of that journal (for instance, a psychiatric nursing journal publishing an article on the treatment of cancer). Determining the reliability of any individual journal article should take into account whether the article has garnered significant citations in higher quality sources, suggesting wider acceptance in the medical literature despite any red flags suggested here.

I suspect that this addition merely writes down what many of our WP:MED editors already do when looking at the reliability of an article. Comments? Yobol (talk) 23:35, 23 July 2013 (UTC)

  • I support inclusion of something of this nature, but the wording may need adjustment ... highlighting specific journals can be tricky. Also for new audiences to the MEDRS page, "predatory" might be defined on first occurrence (you have the link later). SandyGeorgia (Talk) 12:51, 24 July 2013 (UTC)
  • Comments/Questions - 1) The sentences " A red flag that a journal article....reliability of their journal articles." are void for vagueness because they instantly beg the question "What is "predatory" behavior?". 2) The "MEDLINE indexed" thing might be a potentially valuable rule, but we should include instructions for naive users on how to check to see if a journal is MEDLINE indexed. 3) "garnered significant citations in higher quality sources" should be changed to "garnered significant citations in high quality sources" or "garnered significant citations in sources of undisputed reliability". 4) Completely appreciate the difficult question Yobol is trying to address. I'm not super familiar with familiar with WP:MEDRS, but it strikes me that most of the ideas being expressed here by Yobol could probably apply to other scientific literature. Do these ideas not already exist on other policy pages? NickCT (talk) 13:00, 24 July 2013 (UTC)
There is a general policy called Wikipedia:Identifying reliable sources and then there is Wikipedia:Identifying reliable sources (medicine). No other project has written its own guidelines because no other project has had a daily need to send people to a customized explanation of good sources. Blue Rasberry (talk) 18:25, 25 July 2013 (UTC)
  • Comment does this include Elsevier? They have a reputation for publishing fake journals Elsevier#Sponsored_journals. Pubmed indexing says little. This article was half copied from Wikipedia (and they copied wrong stuff) and half copied from a NEJM article [25]. Doc James (talk · contribs · email) (if I write on your page reply on mine) 16:26, 25 July 2013 (UTC)
  • Comment Elsewhere in this discussion some users ask for a definition of "predatory behavior". There is this - Predatory open access publishing. There may be sources which could generalize this article to create an article called Predatory academic publishing practices, but right now, the unregulated predators seem to be using open access platforms because people have confused expectations about how open access works and thus bad people attack researchers from those platforms. Blue Rasberry (talk) 18:31, 25 July 2013 (UTC)

Proposal #2

Thanks for the comments, have attempted to address them below:

An integral part of finding high quality sources is avoiding articles from journals without a reputation for fact-checking and accuracy. A red flag that a journal article is probably not reliable for health claims includes publication by a publisher that has a reputation for exhibiting "predatory" behavior, which include questionable business practices and/or peer-review processes that raises concerns about the reliability of their journal articles. (See "Notes" section below for examples of such publishers.[1][2]) Other indications that a journal article is probably may not reliable is its publication in a journal that is not indexed in the bibliographic database MEDLINE,[3] or the publication of an article whose content is outside the normal scope of that journal (for instance, a psychiatric nursing journal publishing an article on the treatment of cancer). Determining the reliability of any individual journal article should take into account whether the article has garnered significant positive citations in sources of undisputed reliability, suggesting wider acceptance in the medical literature despite any red flags suggested here.

I have removed the mention of specific publishers (redundant with a citation to the more comprehensive list anyways), and moved the description of predatory up. Also added a note on how to determine if a journal is MEDLINE indexed. Yobol (talk) 15:14, 24 July 2013 (UTC)

This doesn't address the problem. Journals don't typically fact-check (which would be a process that editors or peer reviewers may do in part but necessarily completely for an article, or that journalists do at newspapers for all facts). Journals have systems - they then rely on the systems to work, rather than themselves verifying the accuracy of everything that they publish. Not being MEDLINE-indexed does not mean "a journal article is probably not reliable". You would need some pretty strong evidence to support that claim. Citations are definitely not an indicator of quality - indeed, some of the most highly cited publications are those that are egregiously bad. Eg highly disreputable claims about vaccines for example become highly cited as egregious causes of problems. Hildabast (talk) 15:54, 24 July 2013 (UTC)
I agree "probably" may not be the right word, and have changed it to "may". Have also noted that the citations should be positive and not negative to imply reliability (probably common sense, but best to avoid ambiguity). When dealing with reliability issues, we are dealing at its core reputation (as it is based off our core policy of verifiability which discusses reputation of fact checking as of paramount importance). MEDLINE indexed journals have been through a vetting process and would seem to be a good general guide for a discussion of reliability based on reputation (with obvious exceptions on both sides). Note I also included an "out" for any of these red flags with the basis on positive citations so that articles that have been recognized by the medical community can be included. Yobol (talk) 16:20, 24 July 2013 (UTC)
This doesn't address the problem. Still using the term "predatory behavior". As written it would be really hard for a naive editor to apply the policy. NickCT (talk) 12:52, 25 July 2013 (UTC)
I'm hard pressed to think of a credible medical journal that does not subscribe to the ICMJE's Uniform Requirements for Manuscripts Submitted to Biomedical Journals, which includes a code of ethics. Are there any counter-examples? LeadSongDog come howl! 13:20, 25 July 2013 (UTC)
@NickCT:I had hoped a wikilink to our page that describes those predatory behaviors, a brief description in the next clause, and link to two different lists (one peer-reviewed) of said publishers would suffice. How would you improve upon this?
@LeadSongDog: An interesting criteria. Looking at the ICJME website here, however, it may be difficult to use as a criteria as there doesn't appear to be a comprehensive list of journals that follow it, nor does it appear that the ICJME specifically polices their own list vigorously, making this a difficult criteria to apply. (For instance, a journal could claim to follow the requirement but without policing there is no way to verify this). Do you know of a comprehensive list that is actively vetted by someone? Yobol (talk) 13:37, 25 July 2013 (UTC)
(1) just dealing with this list, journals are notorious for adopting standards that they do not consistently live up to. Anyone can say they follow this standard. Equally important, only purely or predominantly medical journals normally list themselves here. Nature & Science & PNAS & PLOS Biology do not, and they all publish articles relevant to medicine. Some journals that do list themselves would normally be considered of very low quality.
(2) more generally, predatory journals do not generally publish significant fraudulent work; anyone wanting to make an impact with such work will publish it in a journal that will be visible--as high ranking as possible, counting on the chance that the reviewers will be lazy; if they don;t succeed with one, they simply send it to another--there is no system by which a journal can know that a ms has been previously rejected. What predatory journals do is publish insignificant work, good or bad, with occasional fairly good work from authors who do not know better.
(3) Beale's list is based on his individual judgement. I greatly respect his judgment, but he has been challenged about failure to remove from his list journals that improve, or dealing adequately with those publishers who have good intentions, but nonetheless are following a model often followed by those who are not as honest. My own opinion is that some of the challenges are correct. No single judge will always be perfect.
(4) There is no foolproof test that everything published in a journal is reliable. The customary test is that people can repeat it--but since both parties may encounter the same hidden artifact, the real test is that people can build a strong structure on its basis.
(5)The best current book about biomed journals is, in my opinion, Philippa Benson and Susan Silver, What Editors Want: an Author's Guide to Scientific Journal Publishing University of Chicago Press, 2013 ISBN 9780226043142 (COI disclosure: I said so when I reviewed the book for Choice). DGG ( talk ) 16:12, 25 July 2013 (UTC)
@Yobol: Yes, Nature Med and the other giants are rather special instances, working to rather more stringent standards than required by the UCM. They have the necessary clout to demand more of their authors. At this list of links from ICMJE we find quick access to some very helpful material. The Council on Publication Ethics (COPE membership include several of the Nature Med family, mostly review journals) provides a formal mechanism for addressing cases. The Council of Science Editors' member journals include the parent Nature, plus Nature Biotechnology, Nature Genetics, and Nature Neuroscience. The CSE has its own formal approach to scientific integrity. I suppose that, in the end, the specific codification is less important than the simple fact of adopting an externally-disciplined code with third-party arbiters. LeadSongDog come howl! 18:26, 25 July 2013 (UTC)
@LeadSongDogInteresting links; though these appear to be suggestions for finding the highest quality journals, not for identifying particularly low quality journals. I wonder if we can incorporate some of this into the last paragraph already present in the Biomedical Journals section. Yobol (talk) 19:45, 25 July 2013 (UTC)
@DGG: Thank you for that additional information. Do you think adding Beall's list is too prejudicial here, or is it a reasonable starting point to look out for journals/publishers to be wary of? I specifically worded this proposal with some ambiguity knowing his list is not in any way definitive. Yobol (talk) 19:47, 25 July 2013 (UTC)
I'm not sure that creating a suitable policy that will always be easily followed by the hypothetical "naive editor" is necessarily an achievable goal. We try to dance around the point, but the truth is that there just isn't, and won't ever be, a perfect recipe that can be blindly followed to reliably determine whether or not a published source is 'reliable'. There are definitely some bright-line rules, but there are also unavoidable swathes of gray.
An individual who comes in without any familiarity with the world of academic publishing is almost always going to trip over an ugly surprise later or – more likely – sooner. What we need from this policy (rather than merely desire) is a clear framework that guides and allows experienced editors (those who come to the table with some knowledge of medical topics and academic publishing) to be able to reach reasonable conclusions about the quality of sources and how they can and should be used in Wikipedia—and to allow them to explain those conclusions to a naive-but-otherwise-competent fellow editor. TenOfAllTrades(talk) 16:45, 25 July 2013 (UTC)
What is a suitable source also depends on what other sources are available. We sometimes use less than idea sources as that is the best there are. We really just want people to use the best available source. I have been discussing issues around this Lancet article in a few other circles.[26]. The "up to 6 h" is not supported by the body of the article specifically [27]. A number of us are of the opinion that they have taken extra liberty with the English language. Additionally there are concerns that the statistical methods used were not appropriate but they used them anyway. Additionally many of the authors receive funding from the company that makes the treatment in question. Doc James (talk · contribs · email) (if I write on your page reply on mine) 18:09, 25 July 2013 (UTC)
I think we need to focus on getting a general guideline for identifying low quality journals rather than getting too deep into specific incidents. Does anyone have any specific concerns in the proposal, or ideas on how to improve it? Yobol (talk) 19:45, 25 July 2013 (UTC)
Suggest
  • When looking for high-quality sources, be wary of articles from journals with a reputation for "predatory" behavior.See: [1][2] Another possible red flag is publication in a journal that is not indexed in the bibliographic database MEDLINE.[4] An article on a topic outside a journal's normal scope may also arouse suspicion (e.g. an article dedicated to cancer treatment published in a psychiatric nursing journal). Note that such red flags may sometimes be counterbalanced by significant positive citations in reputable medical publications that suggest broader acceptance in the medical literature.
    86.161.251.139 (talk) 12:55, 26 July 2013 (UTC)
I'm neutral to your proposed change to the proposal compared to Proposal #2 above; I see merits to both versions, and would be happy with either (one is wordier, but explains the underlying policy reason for this paragraph; the other jumps to the meat of the problem). I do think that your "arouse suspicion" is a better wording, and it splits up MEDLINE indexing to its own sentence. I'm not sure discussion of "reputable" medical publications is a step forward, however, and prefer the one with "undisputed reliability" as it seems less prone to gaming. Yobol (talk) 12:01, 29 July 2013 (UTC)
I agree gaming is a key concern. To my ears at least, "undisputed reliability" seems quite a claim. Would Lancet even pass on that...? 86.161.251.139 (talk) 13:22, 29 July 2013 (UTC)
How about "whether the article has garnered significant positive citations in the highest quality medical sources"? Yobol (talk) 11:53, 30 July 2013 (UTC)
Could you perhaps post another draft to consider? (Fwiw, the incipit "Determining..." sounded rather objective, to my ears at least, for what's always going to be, to some extent, a subjective judgement call.) 86.161.251.139 (talk) 14:33, 30 July 2013 (UTC)
Collapsed human error

MEDLINE indexing - example query

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

So we're comfortable with red-flagging, say, reviews published in the Nature Clinical Practice series (eg [28]) prior to their first being indexed in MEDLINE (from 2009, in what became Nature Reviews)? 86.161.251.139 (talk) 14:14, 27 July 2013 (UTC)
I think the problem is that journals who ceased publication or changed name are not "currently" MEDLINE indexed as they are no longer published. I have added a caveat to the note in my Proposal #2 above discussing this. Yobol (talk) 12:01, 29 July 2013 (UTC)
Sorry, yes you're absolutely right. Strike query! 86.161.251.139 (talk) 13:09, 29 July 2013 (UTC)
The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

Related proposal

See Wikipedia:Wikipedia Signpost/2008-06-30/Dispatches.

MEDRS has enjoyed broad community support over the years partly because it has been well tended, carefully written, and doesn't push unreasonably beyond general guidelines. One thing that might be done to address these newer (and valid) concerns is to write a new Dispatch, which would update the (now) older one, and discuss these new issues. At least it would be a stop-gap measure, and put the info in one place. SandyGeorgia (Talk) 14:30, 29 July 2013 (UTC)

I think having something on the utility of PubMed Health for identifying systematic reviews could be good. But first, isn't it time we specifically mentioned this capability here in #Searching for sources? And if so, how to handle questions regarding the quality assessment filter within the broader context of MEDRS? 86.161.251.139 (talk) 16:23, 29 July 2013 (UTC)
I am not familiar with these Dispatches, what, if any, role would they have here other than being the personal opinion of whoever writes them? Yobol (talk) 11:53, 30 July 2013 (UTC)
They can go into more detail about things like "how to", are sometimes more user-friendly and readable, and I believe that particular dispatch is linked at MEDRS (or once was), because it wasn't just opinion ... it clarified how to use MEDRS in practice. SandyGeorgia (Talk) 13:33, 30 July 2013 (UTC)

Notes

  1. ^ a b c Beall, Jeffrey (2010). ""Predatory" Open-Access Scholarly Publishers" (PDF). The Charleston Advisor. 11 (4): 10–17. {{cite journal}}: Unknown parameter |month= ignored (help)
  2. ^ a b c Beall, Jeffrey. "Potential, possible, or probable predatory scholarly open-access publishers". Retrieved 23 July 2013.
  3. ^ To determine if a journal is MEDLINE indexed, go to this website, and search for the name of the journal. On the journal page, under the heading "Current Indexing Status", you can see whether or not the journal is currently indexed. Note that journals that have changed names or ceased publication will not be "currently" indexed on MEDLINE, but their indexing status, when they were being published, can be viewed under other headings on that same page.
  4. ^ To see whether a journal is indexed in MEDLINE, search for the name of the journal here and locate its entry.

Avoid over-emphasizing single studies, particularly in vitro or animal studies

There is a calculus for weighing the inclusion of single studies.

The title to this section

Avoid over-emphasizing single studies, particularly in vitro or animal studies

suggests part of the calculus: Generally, omit in vitro or animal studies.

The calculus weighs in favor of relying on a single study or a few studies where

  • no secondary authority is readily available.
  • a groundbreaking study comes out that does not contradict reviews, consensus statements, etc.,
  • a well-designed peer reviewed groundbreaking study is the only thing directly on point,
  • a well-designed peer reviewed groundbreaking study is new, and was not available to peer reviewers who prepared other earlier reviews (e.g., a 2013 study was not available to the reviewers who prepared a review published in 2003), or
  • A single new well-designed study that is consistent with and explains prior population studies.

The title "avoid over-emphasizing" aptly emphasizes the judgment process involved in avoiding over emphasis, which is determined by the quality, novelty, and importance of the study itself in the context of the other available authority.

Single studies are not automatically suppressed. That should be clarified, because this section is often cited to imply the contrary. Rather, single studies are used judiciously when needed, especially in the absence of better-established up-to-date authorities. And when single studies are used, the article should hew closely to the researchers own summaries, without violating copyright. Ocdnctx (talk) 16:19, 30 August 2013 (UTC)

Note - the discussion that appears to have prompted this can be found here. Zad68 16:30, 30 August 2013 (UTC)

Dissertations and theses

Presently, WP:RS says "completed dissertations or theses written as part of the requirements for a PhD, and which are publicly available, are considered publications by scholars and are routinely cited in footnotes. They have been vetted by the scholarly community..." Should this language be modified or expanded in any way? See the discussion at Wikipedia talk:Identifying reliable sources#PhD Theses. --Anthonyhcole (talk · contribs · email) 06:23, 25 October 2013 (UTC)

On "primary" sources as a secondary source

An issue (since resolved) on a page I edit came up earlier today and I was wondering what specifically the MEDRS guidance on this was, since it doesn't seem very clear to me.

If a "primary source" (I'm using this term loosely as something which isn't a review of literature) cites another paper to make a claim or reference that has nothing to do with the original research in that paper, is that a reasonable MEDRS secondary source, assuming all other relevant criteria are satisfied? I.e. it's independent of the source being cited, a human study as opposed to an animal study, satisfies WP:MEDDATE, etc.

I realize under those circumstances, that primary source is by definition a secondary source of that material, but I'm not not clear on whether that's sufficient for MEDRS. It seems to be sufficient for RS based upon the material from this page though. Seppi333 (talk) 21:14, 11 October 2013 (UTC)

Too vague to respond to, can you give the example? thx I imagine you are talking about an Intro/background section, or the discussion. Either is a pretty weak secondary source and I would not give it to much weight. Intro, more than discussion (which by definition is building an argument). I would save the weight for a real secondary source... editors are often in a hurry but wikipedia is not... Jytdog (talk) 21:24, 11 October 2013 (UTC)
I'm asking this in complete generality for future reference as opposed to determining if an existing source is ok.
I guess a better question to pose is, under what circumstances is a non-review that is pubmed listed and published in a reliable/well-regarded journal a suitable MEDRS source when the content being cited makes it a secondary source? Seppi333 (talk) 02:24, 12 October 2013 (UTC)
It is difficult to be general. The primary/secondary aspect of a text-source is only one factor. As Jytdog says, the background/info section of a paper isn't as good a secondary source as a proper literature review -- it isn't held to the same standards of comprehensiveness and impartiality by the peer-review or editorial review the paper gets. It is quite likely to cherry-pick the sources that support the theory the primary research is testing (though it is possible a fair writer may cite existing unsupportive research if he thinks the new research improves on it). Another factor is just how controversial or surprising the fact being included is. Something utterly mundane and nearly obvious wouldn't require a strong source. Colin°Talk 11:30, 12 October 2013 (UTC)
I actually wasn't aware of a different treatment between the citations of reviews of literature and non-reviews in peer-review, but in light of that fact I suppose it makes sense to only use reviews in articles. Thanks for pointing that out. Seppi333 (talk) 12:03, 12 October 2013 (UTC)

On a related point, you might want to read WP:Secondary does not mean independent. If Source #2 merely repeats what Source #1 says, it can be independent but still primary. Becoming a secondary source requires some sort of analysis or intellectual transformation. WhatamIdoing (talk) 10:26, 25 October 2013 (UTC)

I already demonstrated my cognizance of that with this quote of myself from above:

If a "primary source" (I'm using this term loosely as something which isn't a review of literature) cites another paper to make a claim or reference that has nothing to do with the original research in that paper, is that a reasonable MEDRS secondary source, assuming all other relevant criteria are satisfied? I.e. it's independent of the source being cited, a human study as opposed to an animal study, satisfies WP:MEDDATE, etc.
— Seppi333

Sourcing at Wikipedia:Fringe theories/Noticeboard#Acupuncture

Over at Wikipedia:Fringe theories/Noticeboard#Acupuncture there is an ongoing discussion concerning the sources used to support claims about the effecticveness of acupuncture. This could really use another set of eyes looking at it. --Guy Macon (talk) 18:28, 31 October 2013 (UTC)

Look right above at Wikipedia_talk:Identifying_reliable_sources_(medicine)#Acupuncture_and_TCM. Blue Rasberry (talk) 13:24, 1 November 2013 (UTC)
Oops. Sorry about that. (Note to self: next time, smoke crack after editing Wikipedia...) --Guy Macon (talk) 16:51, 1 November 2013 (UTC)
=D Lesion (talk) 17:11, 1 November 2013 (UTC)

Reporting effects

Statistical differences are often reported, and sometimes loudly trumpeted, as being the means by which people can determine whether a medical treatment is effective or not. However, a great deal of the papers published, even top-quality ones, have some tremendous errors of interpretation in them. There is an excellent tutorial on this here:

[29].

I would suggest that we exercise extreme caution when promoting statistical results of any kind that are not backed up by multiple independent supporting sources. In particular, psychology studies seem to be suffering a lot from this problem. We should err on the side of not including or even removing sweeping interpretations based on statistical analyses from articles. I think otherwise we run the risk of being very misleading and falling into the very traps outlined at that site.

jps (talk) 13:23, 7 November 2013 (UTC)

Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Identifying_reliable_sources_(medicine)/Archive_7&oldid=1136060000"