CLI Authors’ Responses to Sage Concerns and Retractions

On February 5, 2024, Sage Publications retracted three peer-reviewed articles authored by affiliates of the Charlotte Lozier Institute, including Dr. James Studnicki, Sc.D., M.P.H., M.B.A. and Tessa Longbons. Prior to these retractions, Sage issued an expression of concern against the first of the three articles in July of 2023. The Charlotte Lozier Institute stands by the integrity of its authors and of their groundbreaking, high quality contributions to the science studying abortion outcomes. A full timeline of events and copies of correspondence with Sage can be found at the website assaultonscience.org.

Reproduced below are the authors’ initial response to Sage’s notice informing CLI of its intention to issue an expression of concern, as well as the authors’ subsequent response to the official retractions. As of this publication, seven months and counting, neither Sage officials nor their self-described experts, have offered a single word of response to the CLI authors’ comprehensive rebuttal of concerns expressed about any of these papers. 

Authors’ response to initial expression of concern

Dear Ms. Arpita B:

We have reviewed the email from you outlining concerns expressed about our publication. Following is
our response.

1. Concern expressed: Figures 2 and 3 are “presented with dual y-axis with significant scale differences.” Why was this method used? Comment on why the method may/may not have been a better choice?

Response:

Regarding concerns about our use of dual axis in Figures 2 and 3, it is first of prime importance to note that all the data represented in the graphs are provided in Table 1. This complete report of our findings was sufficient to allow the complaining reader(s) to construct alternative graphs with different axis choices. Any reader is therefore also enabled to regraph the data as they prefer. Therefore, it is obvious that we have fully provided our data in an easy to interpret manner.

Our choice to use the dual axis, in our opinion, provided sophisticated readers a faster way to visualize the degree of change in all three outcome variables over time. The complaining reader’s alternative using a single y-axis compresses the visualization in a manner that, in our opinion, hides important variations over time. More specifically, Figures 2 and 3 displayed the growth in the counts of ER visits following surgical (Figure 2) and chemical (Figure 3) abortions 1999-2015. Each figure arrayed the yearly counts for three categorizations of ER visits: all-cause; abortion-related (ICD-9 630-639); and spontaneous abortion (ICD-634).

Compared to the total ER visit count, the abortion-related and spontaneous abortion categories are relatively small numbers. We wanted to visually illustrate the variation in the two categories with lower counts over time and the differences between them. For example, between 2008-2015, the abortion related visit count following chemical abortion increased from 50 to 790 or 14.8 times the 2008 count. The abortion-related visit count following surgical abortion increased from 313 to 651 for the same period, or an increase of 1.1 times the 2008 count. As demonstrated in the single y-axis version, these differences are not meaningfully discernible visually. The double axis Figures allow a diligent reader to discern the slope differences.

While the specific objection to the two-axis visualization was not made explicit in your expression of concern, it could allow a careless reader to mistakenly perceive abortion-related visits as a larger share of the all-cause visit count. This misunderstanding, however, would be corrected by the immediately following Figures 4 and 5 which directly illustrate the percentages that abortion-related and miscoded spontaneous abortions represent of all emergency room visits following chemical and surgical abortion. The two-axis Figures are appropriately labeled and convey important information that the single axis version does not.

2. Concern expressed: Indicate the rationale behind including all the codes (specifically the reason for including codes 630, 631, 632, 633, and 636). Share your inputs and data extraction.

Response:

The rationale for our study approach and design is discussed in some detail in our paper Introduction. First, prior to our study there was no estimate of the total ER visit burden (incidence) of women following an abortion. Our longitudinal design enabled us to not only track the trajectory of total ER visit morbidity but to also discern the contribution to that morbidity represented by abortion related visits. Thus, there was a simple and persuasive epidemiological reason for tracking all ER visits for this population.

Second, existing research on ER visits following abortion had likely underestimated their incidence for at least two reasons. ER secondary data may describe treatment for a problem (e.g. hemorrhage) which is not, for various reasons, attributed to a prior abortion. Therefore, the ER visit would not be identifiable as abortion related. Also, investigators may only count an ER visit if it meets certain criteria to be considered as urgent or major such as requiring hospitalization, surgery, or a blood transfusion (1). This approach would likely understate the full range of risks associated with abortion.

Unlike most other complication studies, our more comprehensive examination of ER visits was predicated on the existence of a prior abortion confirmed by a paid claim. Our three code categories represent the total ER burden (all-cause), visits related to abnormal (ICD-9 630, 631, 632, 633) or abortive pregnancy outcomes (ICD-9 634, 635, 636, 637, 638), and complications following both (ICD-9 639). Our particular interest in spontaneous abortion (ICD-9 634) derived from statements by abortion providers which encouraged women to withhold information about their chemical abortion when seeking care in the ER (2,3), which could lead to misclassification of treatments as being related to a miscarriage rather than an induced abortion.

3. Concern expressed: “The nature of all ER visits in this study are not provided in the article.” How does this relate to study objective and results?

Response:

All-cause ER visits is the measure for total ER visit morbidity for any reason occurring to women within 30 days of any induced abortion. Chemical abortion is more likely than surgical (OR 1.22) to result in any ER visit. The odds ratios were calculated for the entire 17-year study period, would be larger if calculated for a cross section of more recent years with disproportional chemical abortion growth, and therefore likely understate the current difference between chemical and surgical abortions.

Perhaps the most notable finding regarding all-cause ER visits is that fully one-third of all induced abortions are followed by an ER visit by the year 2015. Whether related to the pregnancy outcome or not, the magnitude of ER use among Medicaid-eligible women in their reproductive years identified in this analysis should be viewed as a health crisis and cause for investigation. The question of “concern” shouldn’t be why we considered all-cause ER visits, but why they are so high! The rate of total ER visits per 1,000 surgical abortions increased 183% between 2002 and 2015. The increase following chemical abortion was even higher at 247%. The likelihood of a chemical rather than surgical abortion-related ER visit (ICD-9, 630-639) during the entire study period is greater (OR 1.53) than for all-cause ER visits. In the period 2002-2015, abortion-related ER visits following surgical abortion per 1,000 abortions increased 315% while the increase following chemical abortion was 507%.

The finding that many ER visits following confirmed abortions are misclassified or miscoded (ICD-9 634) as spontaneous abortions is another seminal finding of this research. To our best knowledge, our paper is the first, and perhaps only reported research to document this serious issue. We found the widest difference between chemical and surgical abortions in their likelihood to be followed by a miscoded spontaneous abortion (OR 1.88). In a follow-up post hoc analysis of this finding, we demonstrated that the miscoded ER visit following chemical abortion represented a significantly increased risk of inpatient hospitalization (OR 2.18) for surgical removal of retained products of conception than did those without miscoding (4).

The tri-partite classification of emergency room visits which served as the study outcome variables are
consistent with the study objectives and demonstrate significant findings consistent with the increased
risk of chemical abortion, relative to surgical abortion, for all three types of ER utilization morbidity. As
per the COPE guidelines, the design and conduct of the study emphasized its importance to existing
knowledge in the domain, clarity in the methods, and originality and innovation in its results and
implications. The history of the paper since its November 2021 publication demonstrates its scientific
quality.

4. Concern expressed: Three papers (Mortensen et al.; Cheung; Sommers) are mentioned to make the point that Medicaid eligible patients may have poorer health and may have a financial incentive to use emergency rooms. The question is whether the behavior of Medicaid eligible women in our paper can be generalized.

Response:

Of course, in the paper we are careful to note that “Medicaid eligible beneficiaries are by definition financially disadvantaged and are not representative of all women experiencing abortion.” We are also careful to identify other limitations of claims data including the fact that services provided to these women, but not paid for by Medicaid, are excluded from the Medicaid files. Coding practices by some providers, insurers and states could also result in undercounting of abortions. All of these facts are appropriately mentioned as limitations in the Discussion section of the paper.

5. Concern expressed: The suggestion is made that the affiliation of some of the authors with the Charlotte Lozier Institute “should have been” revealed considering the “topic of the article.”

Response:

In fact, the affiliations of all authors are documented in the paper. In addition, a bio sketch for each author is included with the paper. Funding support for the research from CLI is also disclosed. Part of the COPE definition of conflict of interest describes “those which may not be fully apparent and which may influence the judgment of author, reviewers, and editors” and “which, when revealed later, would make a reasonable reader feel misled or deceived” (5). All relevant information was fully disclosed.

6. Concern expressed: “The corresponding author is an editorial board member of the journal.”

Response:

Publication by editorial board members is not considered a conflict of interest as long as the member is not involved in the review of his/her own paper. In fact, the practice is explicitly encouraged by most journals as a way to attract and publish high quality content. The SAGE policy: “Editorial board members should be encouraged to contribute articles to the journal, either by submitting their own work (subject to rigorous peer review) or soliciting articles from their colleagues” (6). I have done both for Health Services Research and Managerial Epidemiology. BMC policy states: “Editorial Boards can be encouraged to recommend that their colleagues submit to the journal. They could also be encouraged to submit their own manuscripts, or recommend ideas for commissioned articles” (7). Appointment to the Editorial Board is often recognition that the appointee has a distinguished record of science.

In summary, the “issues” raised in your letter do not present any meaningful challenge to the accuracy or
validity of the findings. Nor do the questions of data visual presentation or design represent any challenge to the methodology or analysis. No errors, miscalculations or deceptive practices are identified. Limitations related to the source of data and the selection of the study population are fully described. No potential conflicts of interest are undisclosed.

We are disappointed that this vague and insubstantial communication would result in the publication of an Expression of Concern before these issues have been fully investigated.

This paper is the single most read in the journal Health Services Research and Managerial Epidemiology. It has been repeatedly referenced in legal cases and legislative discourse in many states. It has enriched the scientific discourse on the relative safety of chemical induced abortion. Most importantly, it is excellent science, and the methods and findings are unchallenged. We respectully ask that you not allow ideologically motivated and unsubstantiated “concerns” to damage the reputation of this work and its authors.

References

1) Upadhyay UD, Desai S, Zlidar V, et al. Incidence of emergency department visits and complications after abortion. Obstet Gynecol. 2015;125(1):175-183. doi:10.1097/AOG.0000000000000603

2) Safe2Choose. Will medical staff be able to notice that I am having an abortion? Accessed September 9, 2021. https://safe2choose.org/faq/medical-abortion-faq/during-abortion-with-pills/will-medical-staff-be-able-to-notice-that-i-am-having-an-abortion

3) Plan C. Abortion pills FAQ: Can I get in trouble for using abortion pills? Accessed September 9, 2021. https://www.plancpills.org/guide-how-to-get-abortion-pills#faq

4) Studnicki J, Longbons T, Harrison DJ, et al. A Post Hoc Exploratory Analysis: Induced Abortion Complications Mistaken for Miscarriage in the Emergency Room are a Risk Factor for Hospitalization. Health Serv Res Manag Epidemiol. 2022;9:23333928221103107. Published 2022 May 20. doi:10.1177/23333928221103107

5) Committee on Publication Ethics. The COPE report 1999: Guidelines on good publication practice. Occup Environ Med. 2000;57(8):506-509. doi:10.1136/oem.57.8.506.

6) Sage. Publishing quality content. Accessed July 7, 2023. https://us.sagepub.com/en-us/nam/publishing-quality-content

7) BioMed Central. Developing and maintaining Editorial Boards. Accessed July 7, 2023. https://www.biomedcentral.com/about/foreditors/journal-information/developing-and-maintaining-editorial-boards

Authors’ response to retraction notice

Attention: Adya Misra and Sage Publishing

The information below responds to Sage Publishing’s notification to us of its intention to retract three of our papers on the topic of abortion. We are scholars and experts associated with Charlotte Lozier Institute and other institutions fully known and identified to Sage when we submitted these articles to you. The first section of this response addresses seven assertions made by a novel set of reviewers selected by Sage and assigned to these papers, two of which have not been subject to any expression of concern and were not identified to us until November 13. The following sections of this response address Sage’s alleged conflicts of interest that are not only inapplicable to our authorship, but have not been applied to any other set of authors publishing on similar topics in Sage’s family of journals.

Sage’s Proposed Retractions are Unjustified

Concern #1: The reviewers stated numerous concerns about the dataset in Studnicki et al1 [Studnicki-1]. The authors count multiple visits by the same patient as multiple visits, thus artificially inflating the number of adverse events. In addition, reviewers note that the emergency room utilization data among the study population was not benchmarked against overall emergency room utilization in the Medicaid population for context.

Response to Concern #1:

If a woman has three emergency room visits on different days within 30 days of an induced abortion, and all are reported, that is not “artificially inflating the number of adverse events.” Rather, it is an accurate count of the number of adverse events. If we were to convert the continuous variable to a categorical yes-or-no visit within 30 days, as the reviewers seem to suggest, the woman with one visit would be equal to the woman with three. That would artificially deflate the number of adverse events. We count certified multiple visits as multiple visits because this approach is both accurate and valid. And we were transparent about this approach: “we identified every emergency room visit occurring within thirty days . . . including multiple visits for each patient.” Studnicki-1, at p. 3.

Research designs like ours, which utilize two experimental groups (chemical versus surgical abortion) and no control group are appropriate and by no means unusual. In the typical design with one experimental group and one control group, the groups are matched to the extent possible for all variations except the intervention. The control group thus enables the investigator to attribute any difference in outcomes to the intervention. The reviewers’ suggestion of “benchmarking against overall emergency room utilization in the Medicaid population for context” is ill-advised for a number of reasons.

First, our comparison of interest was not abortion versus no abortion; it was chemical abortion versus surgical abortion. Second, the “overall Medicaid emergency utilization population” is vastly different from our two comparison groups at least in age and gender, and almost certainly other variables. Third, without an index event (i.e., an abortion), there is no logical way to establish the 30-day at-risk period utilized by the design for an ER visit in the total Medicaid population. Fourth, calculating and reporting an all-Medicaid ER utilization rate would not have affected the chemical versus surgical comparison in any way. Our results would be exactly as reported.

Fifth, because of the differences mentioned, the interpretation of the all-Medicaid rate in the context of our comparison of interest would be irrelevant, useless, and confusing to the reader. Indeed, whether the abortion utilization rate was relatively high or low when benchmarked against the all-Medicaid hospitalization rate is irrelevant to the question of whether chemical abortion leads to higher rates of hospitalization than surgical abortion. And whether the all-Medicaid hospitalization rate is higher than the general population’s rate is again irrelevant because we compared hospitalization rates between chemical and surgical abortions among the Medicaid population (a data set does not exist for the general population).

Concern #2: The reviewers raised the difficulties of using emergency room visits as a proxy for abortion-related complications, or more broadly, an indicator of abortion safety, in Studnicki et al1 [Studnicki-1]. Both reviewers highlight that conflating emergency room visits with complications or adverse events, and without looking at diagnoses or treatments received, may not be a valid or rigorous approach.

Response to Concern #2:

The rationale for our study of ER use following abortion is extensively discussed in the introduction to our paper, and the reviewers did not provide any rebuttal to this explanation:

The emergency room (ER) visit is a particularly insightful event by which to assess and compare the relative safety of chemical and surgical abortions for two reasons. First, adverse events following a mifepristone abortion are more likely to be experienced at home in the absence of a physician, increasing the likelihood of an ER visit. Second, the ER visit can be for any number of complications and is, therefore, a broad proxy indicator for abortion-related morbidity. One major concern is that ER secondary data describes treatment for a condition (e.g., hemorrhage) which may be attributed to a prior event (e.g., abortion), but, as we have seen, the prior event is often missed. For example, a study of abortion-related emergency room visits in the United States, using the Nationwide Emergency Department Sample, categorized whether visits were abortion related based only on information taken from the ER visit record. There was no independent confirmation from a different source that an abortion had occurred. Therefore, a woman who was experiencing excessive bleeding following a chemical abortion but did not reveal the abortion to the ER physician would not be identified as an abortion-related visit. Not surprisingly, the study found an extraordinarily low percentage (0.01%) of abortion-related visits among all ER visits to women age 15 to 49 (7). For all the reasons related to data availability and quality, as well as methodological inadequacies, evidence suggests that postabortion complications are substantially underreported (8,9).

As we have described, research on adverse events following induced abortion varies by procedure, protocols to detect complication, length of follow-up and the sources and quality of data. The emergency room visit as a comprehensive marker for postabortion complications has been infrequently and inadequately utilized in existing research. Therefore, the objective of this research was to complete the first population-based longitudinal cohort study of the trajectory of postabortion emergency room utilization following both chemical and surgical abortions in order to test the hypothesis that chemical abortion results in higher emergency room utilization. We selected a longitudinal cohort design because of its superiority to cross-sectional approaches in suggesting causation. Uniquely, our methodology includes first a confirmation of the actual provision of either a chemical or surgical abortion and, only after confirmation, identifies broadly all emergency room utilization before disaggregating abortion-related ER use. In the absence of a national abortion registry in the United States, this analysis is intended to provide the most comprehensive view of postabortion-related morbidity in the years following the FDA approval of mifepristone abortion, as well as a glimpse of what we might expect in the future.

The reviewer states that we “did not look at diagnoses or treatments received.” That statement is factually incorrect. In fact, we disaggregated ER visits into three code categories to represent the total ER burden (all-cause), visits related to abnormal (ICD-9 630, 631, 632, 633) or abortive pregnancy outcomes (ICD-9 634, 635, 636, 637 and 638) and complications following both (ICD-9 639; e.g., genital tract and pelvic infections, delayed or excessive hemorrhage, damage to pelvic organs and tissues, kidney failure, metabolic disorders, shock and embolism). Our particular interest in spontaneous abortion (ICD-9 634) was derived from statements by abortion providers who encouraged women to withhold information about their chemical abortion when seeking care in the ER (1,2) which could lead to misclassification of treatments as related to a miscarriage rather than an induced abortion.

Thus, our rationale for using the ER visit as a “proxy for abortion-related complications” or an “indicator of abortion safety” was based upon: (1) a comprehensive understanding of the literature; (2) the strengths and weaknesses of various approaches; (3) our desire to produce the first comprehensive appraisal of the incidence of post-abortion ER use for a defined population; and (4) the appropriate level of outcome granularity to answer the primary question of ER use following chemical versus surgical abortion.

The reviewers’ apparent preference for a finer level of granularity in the outcome variables is just that – a preference for an alternative approach. That preference does not challenge the validity of either our methods or results.

Concern #3: In Studnicki et al. 1 [Studnicki-1], the dataset includes years when mifepristone was not registered for use in the United States, which biases any comparison of trends between surgical or “chemical” abortions.

Response to Concern #3:

The reviewers fail to identify the source of the alleged bias from using years 1999-2015 (e.g., the population, the analysis), nor do they provide any examples of how the alleged bias impacts any specific result. In other words, bias is claimed but not demonstrated.

The determination of bias should always be in the context of the specific design of the study and the type of bias to which the design is susceptible. Our design was a retrospective longitudinal cohort study. There is no inappropriate definition of the eligible population (selection bias); no healthcare access bias (selection bias); no misclassification bias (information bias); no detection bias (information bias); and no reporting bias (information bias).

The 1999-2015 observation period does not bias the analysis in any way. In fact, this longitudinal view enables an accurate tracking and analysis of the progression of both the incidence (i.e., counts) of abortions and ER visits, and the rates of ER visits per abortion. We accurately reported the relevant trends for the appropriate time periods—there are no “biases [in] any comparison of trends.”

The data indicate that there were no chemical abortions in the population until 2001 but that chemical abortions grew to 34.1% of all abortions in 2015. That is consistent with the national growth of chemical abortion during that time. Similarly, abortion-related (ICD-9 630-639) ER visits per 1,000 abortions grew for both types of abortions, but the growth in the rate was consistently and progressively higher for chemical than for surgical abortions (Figure 9, Studnicki-1). There is a similar pattern of difference for miscoded spontaneous abortions (ICD-9 634), but the difference for total (all-cause) ER visits is much smaller, actually converging to a similar rate by 2015 (Figures 8 and 7, Studnicki-1).

The numbers speak for themselves. There is no evidence of any bias related to the time period reflected in the analysis. We consider the longitudinal view as a strength, and we are in the process of extending the analysis from 2016-2021. The calculated odds ratios accommodate differences in the volume of the ER visits (99,928 post surgical, 21,355 post chemical) in the observed to expected ratios without bias.

Simply stated, if the outcome of interest is a rate comparison, no bias is involved if no comparison is made.

Concern #4: The authors did not define their outcome of interest that emergency room visits that occur post-medical abortion are miscoded as spontaneous abortion in Studnicki et al2 [Studnicki-2]. A reviewer notes the article’s conclusion that miscoding incomplete abortions as miscarriage is the cause of serious adverse events is inaccurate and unsupported by the data. In addition, the authors did not explain the reasons for assuming all emergency room visits among the study population were miscoded.

Response to Concern #4:

The reviewer’s concern is factually incorrect and nonsensical for at least three reasons. First, the outcome of interest of the post hoc analysis (Studnicki-2) is the rate of hospitalization for women mistakenly miscoded as miscarriage in the ER, subsequent to an ER visit occurring within 30 days of the abortion.

Second, we did not conclude that “miscoding incomplete abortions as miscarriage is the cause of serious adverse events.” Instead, we concluded that abortions miscoded in the ER were more likely to result in hospitalization for any reason (OR 1.06, CL 0.87-1.28) than those not miscoded. We further concluded that chemical abortions are significantly more likely (OR 1.80, CL 1.38-2.35) than surgical abortions to result in hospitalization for the surgical removal of retained products of conception (RPOC), and that chemical abortions miscoded in the ER are more likely (OR 2.18, CL 1.65-2.88) than chemical abortions without miscoding to have a subsequent RPOC admission.

Third, we did not assume that “all emergency room visits among the study population were miscoded.” Women certified as having undergone a completed or attempted induced abortion of a confirmed pregnancy within the last 30 days are highly unlikely to become pregnant again and experience a clinical miscarriage within the same 30 days. These alleged miscarriages are likely incomplete abortions. We logically considered these visits as miscoded.

We did not assert causal interpretations based upon our statistical associations, and it is a purposeful mischaracterization of our papers to suggest otherwise. We instead used words like “association,” “correlation” and “increased risk.”

Concern #5: In Studnicki et al.2 [Studnicki-2], the authors did not transparently report the coding process to outline which codes were utilized to identify hospital admissions. Reviewers note that claims data used by authors are organized around billing events and are likely to include multiple claims per visit, similar to the concern noted in point 1).

Response to concern #5:

This concern is factually incorrect. Every hospital admission following an ER visit and occurring within 30 days of an abortion was identified (n = 4,273 following surgical abortion; n = 408 following chemical abortion). The specific subset of hospitalizations of interest, those considered for the purpose of the removal of retained products of conception (RPOC), was identified using ICD-9 procedure codes 690, 694 and 695 (Studnicki-2, Methods, first paragraph, last sentence).

The claims data included an admission date for each admission. There were no cases where any woman had multiple admissions on the same day. Multiple claims for services occurring during the hospital stay were not included in this analysis, so the issue of multiple claims “per visit” is irrelevant here. This complaint has no foundation in evidence.

Concern #6: Both reviewers highlight concerns with the findings reported in Studnicki et al.3 [Studnicki-3], that the authors do not cite existing literature supporting the hypothesis that hospital admitting privileges for abortion providers do not improve patient safety.

Response to concern #6:

The reviewers’ criticisms about Studnicki-3 are unfounded. Studnicki-3 did not claim to study the relationship between hospital admitting privileges and patient safety. We said, “[t]he objectives of the analysis, therefore, were to describe the characteristics of the physicians who perform induced abortions and to describe the extent to which they hold and use hospital admitting privileges.” We also said, “[i]n particular, the question of whether and how often abortion doctors utilize the ER as a pathway to hospital admission is relevant to the legal issue of requiring privileges for abortionists.”

We did describe the broad consensus among important professional groups (e.g., the American College of Surgeons; the American Medical Association; Joint Commission on the Accreditation of Healthcare Organizations; Accreditation Association for Ambulatory Surgical Facilities, Inc.; American College of Obstetrics and Gynecology; and the American Society for Reproductive Medicine) that credentialing and hospital privileges enhance quality of care and competency. We also suggested, in the Discussion, that “[a]n analysis of physician abortion volume and inpatient admission volume, controlling for important physician characteristics (e.g., board certification), would provide insight into a profile of quality determinants for abortion-related care.” To reiterate, this paper did not attempt to assess the association of hospital privileges for abortion providers with any measure of patient safety.

To our knowledge, there is no study that has reviewed actual hospital admissions by volume and type, by physicians who perform abortions, other than our own study. To our knowledge, a failure to cite someone else’s research has never been the basis for retracting a study based on original data—especially when that other research is irrelevant to the study. This reason is not in the COPE guidelines for paper retractions and is an inappropriate reason to retract this paper.

Concern #7: Both reviewers cite concerns of potential bias in the cohort and analyses reported in Studnicki et al3 [Studnicki-3]. The reviewers note that as physicians often provide abortion services without advertising and are not always required to register with the state, a comprehensive list of abortion providers would be difficult to generate, and that the cohort reported in this study may be biased in a direction unknown to the authors. Further, the study only includes data from the U.S. state of Florida, and does not account for lack of generalizability to other U.S. states due to variation in laws and other factors that may impact healthcare provision.

Response to concern #7:

We emphatically agree that “a comprehensive list of abortion providers would be difficult to generate.” In fact, our paper says, “[m]ore fundamentally, there has been no research at all on the extent to which abortionists actually hold and use hospital privileges. . . . A major barrier to advancing this domain of science continues to be the lack of a universal and comprehensive reporting requirement for all induced abortions and the healthcare professionals who perform them. Valid hypotheses testing analyses of these important research questions will require statistically representative samples of physicians and patients derived from such a comprehensive surveillance system.” Exploratory research is often used when the issue being studied is new or when the data collection process is challenging for some reason, and there is little or no preexisting knowledge or paradigm with which to study it (3).

We made no claims that the state-specific, opportunity sample from Florida was statistically representative, nor generalizable to other states. In fact, important outcomes from our exploratory analysis were areas that we determined (Abstract, Studnicki-3): “Further study of abortionist physicians is indicated regarding their heterogeneous personal and professional characteristics; their career pathways and practice concentrations; their relative integration with or isolation from peers and the professional network; the importance of black and poor induced abortion patients in their total caseload; and, especially for abortionists without hospital privileges, the means by which their patients requiring emergency care and hospitalization are accommodated.”

This self-described “exploratory analysis” developed an innovative approach, utilizing extensive internet and government data, to identify Florida abortionists. This method enabled us to identify 85 physicians whose characteristics and use of hospital privileges we were able to profile. This research provided insight into how this domain of research might be improved in the future. This was not a hypothesis testing study. It is a superb example of exploratory research.

Summary of the Seven (7) Concerns:

The reviewers claim to have “highlighted fundamental concerns with the study design, methodology, assumptions about healthcare indicators and analyses, such that the conclusions may not be adequately supported by the results.” The letter from Sage’s lawyer (R. Sander, November 21, 2023) further states that these “substantive findings by the reviewers were most significant in the determination that
retraction of the article was necessary under COPE guidance [sic].”

An objective review of each concern and our responses demonstrates clearly that:

1) No single specific finding in any of the three papers has been explicitly challenged, let alone invalidated.
2) There is no evidence of a major error, miscalculation, fabrication, or falsification.
3) There is no breach of any of the COPE guidelines that could permit Sage to retract any of our published papers.
4) The retraction of any of these papers, let alone all three, is demonstrably unwarranted.

Sage’s Proposed Retractions Violate COPE Guidelines

Sage represents itself as an adherent to COPE guidelines. Yet a review of COPE retraction guidelines  reveals no justification for this decision to retract any of the three papers, much less all three.

First, there is no allegation—much less evidence–that our findings are unreliable, “either as a result of major error (e.g., miscalculation or experimental error), or as a result of fabrication (e.g., of data) or falsification (e.g., image manipulation).” Our results are replicable and accurately reported.

Second, retraction is not appropriate if “[t]he main findings of the work are still reliable and correction could sufficiently address errors or concerns.” Retraction also is not appropriate if “[a]uthor conflicts of interest have been reported to the journal after publication, but in the editor’s view these are not likely to have influenced interpretations or recommendations or the conclusions of the article.” Notably, the editors of the retracted articles have not articulated any apparent “competing interests” (beyond those already disclosed) that could have adversely influenced the interpretations, recommendations or conclusions of the articles.

Third, as detailed above, our data is accurately reported and our interpretations, recommendations and conclusions are narrowly stated and limited to the actual findings reported. Our conclusions, as noted above, do not declare causality. They simply and accurately summarize statistical associations that warrant further investigation. The original reviewers and editor agreed.

Lastly, our affiliations are in fact already noted in the paper. Normally, identification of employment with an affiliated institution is considered adequate notice, since clearly any reviewer or reader can easily learn more about us and our institutions. There can be no credible allegation of deception against us. And readers have full notice of our affiliations upon reading the paper. The term “Pro-Life” even appears in the name of the association with which Dr. Donna J. Harrison is disclosed to be affiliated: “American Association of Pro-Life Obstetricians and Gynecologists, Eu Claire, MI, USA.”

Sage’s Proposed Retractions Misrepresent ICMJE Disclosure Standards

The retraction notice implies that we should have completed and provided the ICMJE Disclosure Form. But we completely and accurately disclosed all affiliations and financial support as part of the online submission process, and we substantively complied with all ICMJE recommendations by doing so. More importantly, the information we provided to Health Services Research and Managerial Epidemiology clearly “assist[ed] the Editor in evaluating whether sufficient disclosure has been made within the Declaration of Conflicting Interests provided in the article,” per section 4.5 of the Journal’s submission guidelines. And the Editor correctly concluded that our disclosures were sufficient.

Further, only line item 13 of the ICMJE Disclosure Form, asking for “Other financial or non-financial interests,” could be remotely associated with a disclosure regarding an author’s informed opinion regarding the safety or risks of induced abortions, religious affiliation, political party, voting records, or other issues related to abortion. But historically, such information has never been required by any other publisher or journal with which we have, collectively, published hundreds of studies. Nor is it common practice of journals to require the thousands of authors publishing studies on controversial issues in general to describe any political, religious, or philosophical positions of their own, their employers, or institutions with which they are affiliated. Political and religious litmus tests, in particular, have never been required for academic research, nor should they.

Per normal practice, no potential conflicts were undisclosed. Our institutional affiliations were clearly stated, and the nature of our roles at our respective institutions were noted in our author biographies. The mission statements and activities of these institutions are publicly known and easily discoverable.

In its retraction notice, Sage alleges that “[t]he authors are all affiliated with ‘pro-life political advocacy.’” But in fact, neither the authors nor their affiliated institutions are involved in “political advocacy.” All three organizations are limited, by law, to educational and research efforts.

Furthermore, if Sage now purports to have always required ideological disclosures, Sage itself has not maintained that standard it seeks now to impose on us. Despite Sage’s dismissal of this point as “unsupported,” a cursory search clearly demonstrates that Sage journals have published dozens of articles on abortion access and safety by researchers affiliated with organizations with very public and open positions on abortion. For example:

•  A study on abortion safety in which the sole author is affiliated with Guttmacher Institute, “a leading research and policy organization committed to advancing sexual and reproductive health and rights (SRHR) worldwide,” which includes “safe abortion care” as “essential.” No conflicts were disclosed.
• A study on abortion methods in which all authors are affiliated with Ipas, which collaborates “with partners around the world to advance reproductive justice by expanding access to abortion.” No conflicts were disclosed.
• A study on telemedicine abortion in which the authors are affiliated with Ibis Reproductive Health and Advancing New Standards in Reproductive Health (ANSIRH). Ibis commits to “using research to advocate for improved, client-centered abortion care,” while ANSIRH maintains that “abortion is an essential part of reproductive health care.” No conflicts were disclosed.

In any event, the COPE guidelines for undisclosed conflicts of interest recommend publication of a new statement of any potential conflict of interest as an alternative to retraction, an avenue that Sage has failed to pursue here.

Additionally, Sage had and has entire and exclusive control of the review process including the selection of the reviewers, who are unknown, as is appropriate, to us. The retraction notice declares that one of those reviewers is identified as an associate scholar of the Charlotte Lozier Institute (CLI). These scholars are generally not employees of CLI. They are simply scholars who generally agree with or support CLI education and research efforts. We did not select the reviewer at issue, and the reviewer’s identity is still unknown to us. In fact, this reviewer may not have been a CLI associate scholar at the time of his/her review. Likewise, because Sage practices double-blind peer review, the reviewer was equally unaware of our identities when evaluating our work. Sage does not indicate how the scholar would have known that the studies that he or she reviewed had been authored by CLI researchers.

Additionally, there is no assertion made, or reason to believe, that this scholar’s review was inappropriate or inaccurate. If there was a breakdown in the peer-review process, which we have no reason to accept, it is entirely the responsibility of the journal. Equally important, the other peer reviewers of our studies had no concerns with our research. Sage has not raised any issues with these reviewers, and their review confirms the quality of our studies. That’s one of the benefits of having multiple peer reviewers (4).

In short, these complaints are substantively and substantially ungrounded. Moreover, individually and together, none of these complaints, according to COPE guidelines, justifies retraction of even one of our articles, much less all three.

Final Notice

Finally, we wish to make clear that while we have provided this document to you in accord with your request for response prior to publication of your retraction notice, this response is not a waiver of our rights to seek compensation for the publication fees we paid under contract and in good faith, based on a representation that Sage had conducted a thorough and definitive peer review prior to publication, and for any financial and irreparable damages these retractions have caused or may cause to our reputations and careers.

We request Sage’s prompt response to this document in the interest of correcting the public record and limiting further damages to CLI.

References:

1. Safe2Choose. Will medical staff be able to notice that I am having an abortion? Accessed September 9, 2021. https://safe2choose.org/faq/medical-abortion-faq/during-abortion-withpills/will-medical-staff-be-able-to-notice-that-i-am-having-an-abortion

2. Plan C. Abortion pills FAQ: Can I get in trouble for using abortion pills? Accessed September 9, 2021. https://www.plancpills.org/guide-how-to-get-abortion-pills#faq, Google Scholar.

3. George T. Exploratory research: Definition, guide, and examples. Updated November 20, 2023. Accessed November 27, 2023. https://www.scribbr.com/methodology/exploratory-research/

4. “Peer review best practice,” https://us.sagepub.com/en-us/nam/peer-review-best-practice#:~:text=In%20general%2C%20Sage%20strongly%20recommends,borderline%20manuscript %20may%20require%20a; “Why Use More Than One Peer Reviewer?”, https://www.wiley.com/en-us/network/publishing/research-publishing/submission-peer-review/why-use-more-than-one-peer-reviewer.

Studnicki-1: Studnicki J, Harrison DJ, Longbons T, Skop I, Reardon DC, Fisher JW, et al. A Longitudinal Cohort Study of Emergency Room Utilization Following Mifepristone Chemical and Surgical Abortions, 1999–2015. Heal Serv Res Manag Epidemiol 2021;8:233339282110539. https://doi.org/10.1177/23333928211053965.

Studnicki-2: Studnicki J, Longbons T, Harrison DJ, Skop I, Cirucci C, Reardon DC, Craver C, Fisher JW, Tsulukidze M. A Post Hoc Exploratory Analysis: Induced Abortion Complications Mistaken for Miscarriage in the Emergency Room are a Risk Factor for Hospitalization. Health Serv Res Manag Epidemiol. 2022 May 20;9:23333928221103107. doi: 10.1177/23333928221103107. PMID: 35633832;
PMCID: PMC9130799.

Studnicki-3: Studnicki J, Longbons T, Fisher JW, Harrison DJ, Skop I, MacKinnon SJ. Doctors Who Perform Abortions: Their Characteristics and Patterns of Holding and Using Hospital Privileges. Health Serv Res Manag Epidemiol. 2019 Apr 15;6:2333392819841211. doi: 10.1177/2333392819841211. PMID: 31020009; PMCID: PMC6466461.

Learn more about the retracted papers and CLI’s response at www.assaultonscience.org.

Editor’s Note: Light formatting edits were made to the original responses to improve readability. The original formatting may be viewed in the PDFs at the Print/Download button above.