Developing Best Practices for Supplemental Materials Linda Beebe June 2, 2011 SSP June 2, 2011

C OLLISION OF 2 W ORLDS

SSP June 2, 2011 Explosion of─  Research  Data  Accrued Knowledge Increased Requirements  Funding Bodies  Reporting Standards

Supplemental Materials—it sounded like such a good idea. The author could expand on their research. Science would be better with data needed to verify or replicate study at little additional cost. We could enhance reporting of science with multi-media. We looked to technology to solve problems—but kept our print-centric views. And we did it on our own— no standards or best practices. SSP June 2, 2011

Outcomes for the author? SSP June 2, 2011  May get to showcase new work that would not otherwise be seen.  May also risk displaying weak work that otherwise might not be seen.

Outcomes for the user? Lack of descriptive metadata Discoverability issues Lack of context Concern about persistence No clarity on citations Some mystery in the main article about what is supplemental ─ a maze, maybe not value- add taken as a whole. SSP June 2, 2011

Outcomes for the publisher? Direct costs Diverted energies— already crisis in peer review Tough decisions─ What is value-add? Peer review dilemma, quality vs workload? Plan for migration? SSP June 2, 2011

NISO-NFAIS Working Group SSP June 2, 2011

Business Working Group Co-Chairs: Linda Beebe & Marie McVeigh Define Supplemental Materials, structurally and functionally. Define related terms, such as data, citation, and article. Recommend methods of referencing and linking to and from supplemental material and for providing context. Recommendations around metadata, persistent identifiers, and citations. Recommend processes for peer review, production, and curation. Consider permissions and accessibility issues. Recommend responsibilities for authors, editors, peer reviewers, publishers. SSP June 2, 2011

Technical Working Group Co-Chairs: Dave Martinsen & Sasha Schwarzman Recommend metadata, persistent identifiers, and granularity of markup needed to support practices recommended by the Business Working Group (BWG). Recommend supports for referencing and linking to and from Supplemental Materials and for handling cited references within Supplemental Materials. Recommend processes for archiving, preservation, and forward migration of various types of Supplemental Materials. Recommend processes for packaging, exchange, and delivery of Supplemental Materials, taking into account variations in the location and hosting of those materials. Recommend technical support for accessibility practices recommended by the BWG. SSP June 2, 2011

On Different Wave Lengths Disciplines vary in use of supplemental material. Differ in style systems and culture. Readers vary in need for information—some current awareness, some deep digging. Different approaches to underlying data. Very different approaches to delivery systems. Technology enabled, but still using print. SSP June 2, 2011

Data One Type of Supplemental Example of evolving ecosystem. Print world—no datasets part of article. For some, almost synonymous with supplemental. Journal articles—indeed whole journals—devoted to data emerging. For these data are integral content. Management of data in general not within scope of recommended practices. Address inclusion of data when published as supplemental (with a little aside on sharing). SSP June 2, 2011

Illustrates Discipline Variances Some publishers—such as AAAS and ACS—require posting of data in a publicly accessible repository for replication. Some publishers—such as AGU—identify acceptable repositories. Some publishers—such as APA—currently say only that authors should provide data to researchers for verification. SSP June 2, 2011 More calls for transparency

Data Sharing—the ideal & reality What they say: PARSE study—84% of scientists think it useful to link data to articles. In Psychology, 80% say they share their data. 2008—Harvard faculty voted to require faculty to deposit data in Harvard repository. What they do: Only 25% said their data are openly available. Only 20% actually do. By 2011, only a fraction have done so. SSP June 2, 2011 Most ethics codes call for some level of sharing.

Reasons for not sharing─ It takes time, may require extensive explanations of coding or just plain clean-up. I’m not finished—I can get more articles. Who will curate/protect it? Will I be credited? They have several fears— Loss of confidentialty Potential harm to subjects Potential faulty re-analyis May be proven wrong Loss of control SSP June 2, 2011

What we are saying... Collaborative sharing best practice. Requires clear metadata and explanations. Professional ethics around secondary analysis. Particular concern for studies with human participants. SSP June 2, 2011

Multi-Media Another Example Five years ago, audio or video not possible in the article. Today still generally supplemental. BUT some now incorporating in PDFs. Executables as part of the article? Expect much more interactive content. SSP June 2, 2011

Supplemental Today, Not Tomorrow “... over time the concept of supplemental material will gradually give way to a more modern concept of a hierarchical or layered presentation in which a reader can define what level of detail best fits their interests.” −Emilie Markus, Editor-in-Chief, Cell Article of tomorrow may be linked chunks, not a narrative. SSP June 2, 2011

Challenges for Publishers Expectations  Quality uber alles—  Peer review all.  Edit to same level as article.  Maintain all links.  Assure migration. Limiting Factors  People resources  Financial resources  Technical resources SSP June 2, 2011

We need some order now. SSP June 2, 2011

Defined 3 Types of Content Integral. Critical to understanding the work reported, but technical issues prevent inclusion in the framework. Additional Content. Expansion of core article, added detail and context; provides layered approach for readers. Other Related Content. Content may add to the understanding or enable replication; generally hosted by others. SSP June 2, 2011

Recommended Practices Selecting Editing Assuring Findability Citing SSP June 2, 2011

What we are saying... SSP June 2, 2011 Selecting Review same level Useful, relevant, not file drawer Editing Publisher/Editor determine. Provide notice if not. Assuring Findability Consistency Online TOC Reference Indexing Coverage Don’t hide! Citing Within article, cite & link as for a table. Not in reference list for integral.

More Recommendations Providing Links & Context Preserving Assuring Accessibility Rights Management SSP June 2, 2011

What we are saying... SSP June 2, 2011 Links & Context Bi-directional if possible Links must work! Context is essential. What is this? Why here? Preservation Integral same level as article Clarity on what can do. Encourage authors to deposit elsewhere also. Accessibility Should be same level as article. Strive for ideal, recognize difficulty. Rights Management Treat rights same way do for the article. No authority for Other Related Content.

2 Working Groups, 2 Roles BWG—What? TWG—How? SSP June 2, 2011

TWG Working Group Task Forces Metadata—have strawman DTD Linking and persistent identifiers Packaging and exchange Preservation and archiving Accessibility SSP June 2, 2011

Next Steps Early summer share BWG Sub- groups over the summer TWG Fall meld and refine Both SSP June 2, 2011

Final Set of Practices Approved by NISO and NFAIS. Shared with the community and refined. Serve as a temporary roadmap. Living document for rapidly changing environment. SSP June 2, 2011

We welcome ideas! NISO— To see working groups: Also join the Business Stakeholders’ Group at that page. NFAIS— SSP June 2, 2011

T HANK YOU ! SSP June 2, 2011 Linda Beebe Senior Director, PsycINFO American Psychological Association