Basic features

Main features include:

• Workflows: Kotahi supports multiple workflows including PRC (Publish, Review, Curate) and the submission, peer review, and publishing of everything from preprints, micropubs, journals, and books to exciting new types of publishing.
• Review models: Kotahi enables publishers to use a number of different review models: open, blind, or double blind. Reviewers can have individual reviews or collaborate on a shared review.
• Editing: Kotahi is designed around the idea of editing and authoring articles in the browser, using Wax, Coko’s full-featured web-based word processor.
• Realtime updates and chat: Kotahi is built to get the best from web collaboration and supports real time notifications, chat, synchronized updates and more. Live chat (both text and video) functionality can be used to communicate with authors or collaborate as an editorial team.
• Metadata: The in-browser form builder can be used to design submission forms to capture and manage metadata specific to each journal’s workflow..
• Single-source methodology: Kotahi supports a single source publishing workflow where all stakeholders are able to access the same manuscript during content creation (peer review, editing, etc.) and production (copyediting, semantic tagging, etc.). This obviates the need to track and control documents across multiple systems.
• Versioning: Versioning is supported so that changes to a submission can be easily inspected.
• Exporting: Currently Kotahi exports to PDF, JATS, and HTML. A GraphQL API makes it possible to connect Kotahi’s data to other systems (like PubMed) as needed.
• Integrations: Authorization is through ORCID; more integration with ORCID in the future is possible. Kotahi can register DOIs for articles via integration with Crossref. It can also publish reviews of existing articles as Hypothes.is annotations, as used by eLife for generating Transparent Review in Preprints (TRiP) on the bioRxiv server.
• Reports. Admin users of Kotahi get a dashboard showing the progress of articles through the system.

Users

Development of Kotahi is being supported by eLife, Amnet Systems, the Journal of Medical Internet Research (JMIR), and Aperture, an open publishing platform from the Organization for Human Brain Mapping. JATS capabilities have been added in conjunction with Amnet, a publishing solutions company in India; they’re using Kotahi in their workflow (Reach) to produce JATS.

The Coko Foundation

The Coko Foundation was founded with seed funding from founder Adam Hyde’s Shuttleworth Foundation fellowship. It produces a variety of open-source software for publishing.

Kotahi builds on work done for another flagship project from Coko, Editoria. Editoria (https://editoria.community) is a fully fledged post-acquisition book production workflow developed by the Coko team in collaboration with the University of California Press (UCP) and the California Digital Library. Editoria is now in live use by UCP, ATLA Press, punctum books, M Publishing, UNC Press/Longleaf Services and Book Sprints. Kotahi’s development will be contributing back to Editoria, as that project will soon be outputting BITS.

Coko frameworks used by Kotahi

Kotahi builds on a number of different projects produced by the Coko Foundation. Major components include:

• PubSweet (https://pubsweet.coko.foundation) . PubSweet is Coko’s open source framework for building state-of-the-art publishing platforms. Kotahi uses this as a component library and parts of its framework.
• xSweet (https://xsweet.org) . A high fidelity MS Word (.DOCX) to HTML converter. XSweet extracts the contents of MS Word documents from their underlying XML into HTML, which can then be edited as text inside of Kotahi; inline images go into Kotahi’s image store. XSweet also pulls metadata from manuscripts.
• Wax (https://waxjs.net) . A web-based word processor with a wide variety of features which can be configured in a number of ways. Wax is at the heart of Kotahi; the Kotahi version of Wax allows marking up of documents in a way that can be easily translated into JATS.
• Paged.js (https://pagedjs.org) . Paged.js is a free and open source JavaScript library that paginates content in the browser to create high quality PDF output from any HTML content. Using instance-specific templates, this allows Kotahi to export manuscripts and metadata as print-ready PDFs.

In conjunction with Kotahi and Editoria, work is also being done on another Coko project:

• Flax. Flax is a publishing front end, a web presence for content produced in Kotahi and Editoria. Flax is in active development as a way to present content which lives in Kotahi in a web-first model.

And two other projects have ended up at Coko, which may be integrated with Kotahi in some way:

• Science Beam uses machine learning to convert PDF scholarly articles to XML with high accuracy. Originally built by eLife and transferred to Coko to further develop and maintain.
• Libero Editor is a JATS XML editor (a replacement for Texture). Originally built by eLife and transferred to Coko to further develop and maintain.

While traditional workflows involve direct editing of documents inside of Kotahi, it’s also possible to use it to run a PRC workflow, ingesting preprints from external servers. In that particular case, ScienceBeam might be used to generate JATS from PDF-based manuscripts.

Workflow

The Kotahi workflow is very configurable and can support workflows that range from very linear, to very concurrent. Submission types are also configurable. These two features together mean that Kotahi can support a wide variety of use cases and workflow models. As an example, here’s a diagram of how Kotahi is being used by the Aperture Neuro journal (Organization of Human Brain Mapping):

This diagram doesn’t need to be explored in depth for an understanding of how JATS fits into Kotahi (a link to a thorough explanation is provided in the appendix), though it’s useful for thinking about how documents travel through a system from user to user. Many different people are viewing a manuscript—in different states—and may be making changes; and every publication probably has their own, slightly different, system. Aperture, for example, is publishing to static HTML pages as a one-time event; but you could also design a system where publication happens multiple times. Most of the time we imagine JATS as being the final point in a document; but Kotahi is designed in such a way that it doesn’t have to be.

Fig. 1Workflow diagram by Henrik van Leeuwen with input from Ryan Dix-Peek

HTML

Coko is using HTML as the source of truth. This is in part because Coko started with the web has a given; the web is built on HTML, CSS, and JavaScript, which are, by and large, the tools that Coko’s software is built on – using node.js server-side. However, we’re also realistic about the ways in which authors are working: they’re largely using Microsoft Word to prepare manuscripts—and editors by and large seem to resist moving away from Word.

Attempts to “get out of Word” have tried to jump from unstructured Word to very structured XML formats by:

1. copying over all the data in the document and
2. interpolating structure at the same time in an attempt to “understand” the “intent of the author” (or a proxy) as represented by the display semantics of the document.

But if the structure does not exist in the first place, you have a problem. Coko’s xSweet (developed by Adam Hyde and Wendell Piez) Word-to-HTML conversion retains step one (copying over all the data) and replaces step two (interpolating structure) with a process that forsakes the introduction of any structure in favor of carrying over whatever information from the original Word file might be useful later on, whether for programmatically understanding or manually applying document structure. The best solution of course, being a little bit of both. Hence we:

1. convert the unstructured Word document into an unstructured (or partially better structured) HTML document; and
2. interpret the original Word file and carry forward as much information as that original Word file contained for possible future use in interpreting the document structure—or representing any features of interest—while not actually structuring the document.

Interestingly, since HTML does not require you to enforce a strict document structure if you do not have it, an unstructured document flows into HTML as easily as a structured, or partially structured, document flows into it. If your aim is a well-controlled document format, such a failure to enforce strict structures is regarded as a flaw or weakness. Yet, since we do not have much or any document structure in the originating Word document, and our goal is to improve it—HTML’s flexibility becomes a critical feature.

This process produces an intermediary carrier format—an information- rich, sanitized HTML format that is suitable for improvement. In an HTML editor, we can then bring to bear further tools for adding structure, reviewing, commenting, and revising.

Single-sourcing

Kotahi’s design is also informed by the basic idea of single-sourcing. There’s a lot that could be said about this; where it’s relevant to Kotahi and JATS is that the Kotahi model sees JATS production as an integral part of the publishing workflow, not something that should be considered as a separate step.

Publishing has long suffered from broken workflows that slow down the time to publish and unnecessarily inflate the cost to publish. In general, this comes from a historical disconnect between content creation processes and production processes.

In many publishing environments, authors, copy editors, proofers etc. create and improve content primarily in one tool (usually Microsoft Word). The content creation process feeds into the production process where production staff create a multitude of other formats—HTML, PDF (for print and screen), as well as XML and ebook formats.

To convert to these file formats, the content has to either be programmatically converted to various target formats via software (e.g. Pandoc or bespoke software), or manually converted by people using software (e.g. InDesign). Publishers achieve this by either slinging the content “over the wall” to a publishing services vendor or contractor, or they employ internal staff. The people doing these conversions belong to the general category of “production people”—usually programmers, designers, or “format wranglers.”

Workflows that look similar to the above—most publishing workflows—separate the content creation from the content production. This disconnect separates who does the work, the tools used, and, most importantly, what files are worked on. The people, the tools, and the working files all change as the content jumps from content creation to content production.

A simple journal example: imagine that the production staff have discovered from proofs that figures are in the wrong place. This information must be communicated to (for example) the publishing services provider (via email, MS Word, annotated PDF, etc.). The vendor interprets the information, makes the changes in their (various) tools, and sends it back to the publisher to check. This may be iterated repeatedly.

Larger publishers have staff to manage the communication of changes, track the changes, check the changes etc. This costs time and money. If publishers don’t do this well, the consequence is that more errors are introduced.

This is the problem single sourcing is meant to solve. Single sourcing is a general approach to publishing workflows that is intended to avoid disconnecting the content creation and production processes—saving time and money, and reducing errors.

Single sourcing isn’t a specific solution, it is a general idea that must be intentionally designed into a publisher’s workflow. Single sourcing changes how people work and often requires a different tooling. The secret really, if we zoom out to a high-level abstraction of the problem, is to work out how the content creation and production people can work in a shared environment where they all work on the same files, the same source files—hence the term “single source.”

In a single source environment, if a change needs to be made while the content is in production, the content people can make the change themselves. Less time and communication required, less to-and-fro, less management overhead, and fewer errors.

What this also allows is moving from a more linear model of publishing—a manuscript goes from step A to step B to step C, each done by a separate person—to a model that allows more concurrency and collaboration, leading to more efficient document production.

There are certainly ways of producing JATS that are more thorough-going than Kotahi’s: Kotahi only uses a tiny fraction of what can be done with JATS. The virtue of Kotahi’s approach to JATS production—and perhaps why it is valuable to the wider landscape of JATS—is that it makes the threshold to generating JATS very low. Publications that might have thought that JATS was too complicated might start making it part of their workflow because it requires very little effort.

JATS in the production editor

The production editor also includes features related to JATS functionality. Citations can be tagged as such; material can be grouped into appendices; acknowledgments can be treated as such.

The design principle behind the production editor is not to expose the JATS model; rather, the conversion to JATS automates as much of the work as is possible. (The user, it should be said, doesn’t have any knowledge what the internal format is; they only see content and formats.) For example, internally a document might have a typical flat Word/HTML structure like this:

Fig. 2

The Kotahi concept is to let users structure content as they do in Word, given that they’re probably more used to Word—there is no outline viewer of a document’s structure for the user to organize. But on conversion, that flat structure is carefully wrapped in JATS-style sections from the smallest header up, becoming this:

Fig. 3

This is a more logical structure; but the user is not forced to convert the document into the nested format. The user imposes structure visually; the converter attempts to make sense of it.

A similar approach is currently taken with citations. While we anticipate that a fully-functioned citation editor may eventually become part of Kotahi, we’re currently dealing with manuscripts with bulleted or ordered lists of citations. The Kotahi approach would be to wrap the citations in a Reference List element; every paragraph or list item that’s inside a Reference List in Wax becomes a <mixed-citation> on export to JATS. There’s currently nothing stopping a user from putting a Reference List or an Appendix element in the middle of a document; on conversion to JATS, Kotahi pulls the reference list or appendix out of the body text and moves them to the back matter of the resulting JATS file. The idea we’re working with is to prevent the user from making invalid JATS as much as we possibly can. An area that’s currently being worked on is how to handle the front matter for documents. JATS structures all of this very carefully in the <front> tag; we can generate this very easily from form data, which, for example, might split a list of authors into neatly structured first and last names and affiliations. We want to present this material as WYSIWYG front matter at the top of the document in Wax; right now, anything that’s wrapped in a Front Matter element gets ignored by the JATS conversion, but this should be more robust. Wax supports complex plugins for entry; the way we’d like to handle this is by having an instance-specific template for front matter (very similar to the Nunjuks templating that’s used to create HTML and PDF versions of documents) that would allow direct editing in the editor.

Who’s doing this?

One part of the Kotahi philosophy that should be noted: there’s an assumption that the production editor is not necessarily expert in JATS, though that person might have some domain knowledge. Kotahi is aimed at making JATS at scale; a journal might publish thousands of articles per year. Articles published by the same source tend not to have a huge amount of variation; the JATS that comes out of them will tend to look fairly similar. While it will take a little time for the production editor to learn how content needs to be formatted in JATS-flavored Wax, the learning curve should not be steep.

Validation

When JATS is created, it goes through an XML validator and is validated against the JATS schema. Because the number of elements allowed in Wax has been carefully enumerated and constrained—all of the paragraph styles, list styles, and character styles that can appear in Wax have been defined by the editor’s configuration rather than the infinite possibility of HTML and CSS—the range of the JATS spec that those elements can be converted to is relatively small and can be thoroughly tested. Validation errors are not something that users should see; instead, they tend to indicate that something is wrong on the development end.

As previously mentioned, the production editor allows users a wide amount of flexibility with JATS elements. There’s nothing stopping a user, for example, from putting in 12 abstract sections, even though a JATS document can only have a single <abstract> tag. Currently if you have a manuscript in the production editor with 12 abstract sections, the first will be pulled out of the body and moved to the front matter of the created JATS and put inside an <abstract> tag. The others will be deleted as abstracts can’t appear in the body. The result is perfectly valid JATS; though this result might be confusing to a novice user.

The plan going forward is to offer structural hints to the user while they are in Wax: so if, for example, a second abstract element is inserted into the text, it might get a red border, and a warning might explain that a document can only have a single abstract in JATS. (Kotahi won’t keep you from inserting two abstracts: while JATS disallows multiple abstracts, there’s nothing to stop you from making a PDF with multiple abstracts if you want.)

JATS as part of the document lifecycle

JATS export (as well as PDF export) is currently on-demand from within the production editor. Conversion to JATS is not a one-way process; although it would most likely happen near the end of a document’s workflow, an editor could generate a JATS file, inspect it, make changes to the manuscript, and regenerate.

Crucially, a manuscript that’s gone to the production editor and had JATS elements attached to it in Wax can be sent back to the non-production editor; so if a last-minute mistake is discovered that needs to be corrected by an author, the author can work in the same editor that review was done in, rather than using a separate (and possibly confusing) tool.

Initial setup: metadata

The front matter of a typical JATS file composed from an article contains major types of metadata: metadata specific to the article and metadata for the journal (the journal’s title, ISSN, etc.). Journal metadata (with a few exceptions) tends to be similar across all articles in a journal; this is set up as part of the process of getting an instance of Kotahi off the ground. (Initial setup similarly involves setting the stylistic defaults and layouts for articles as they are seen in the editor and exported as HTML and PDFs.)

The shape of article metadata will vary from journal to journal. Kotahi’s interactive form builder allows setting up the sorts of metadata that will be attached to each article. However, there’s not a one to one correspondence between what’s in the form and what’s exported as metadata. A form may include workflow-specific metadata—who, for example, anonymous reviewers are, something that some users of Kotahi but not others should know – that isn’t part of the metadata that’s involved in JATS. This is controlled by a mapping of form fields to JATS metadata types; this would typically be done during the setup of a new instance of Kotahi, but might change over time.

A caveat about images & delivery formats

Until very recently, Kotahi handled uploaded images in a document as base–64 strings. This is in the midst of changing, and this won’t be the case next month. If images are handled as base–64 strings, however, JATS can be exported as a single XML file, which is the current situation. A full-function image store is currently being built; when this is in place, what’s output as JATS will not be a single XML file but rather a ZIP file containing the XML and the images in multiple formats—if authors have TIFFs as their original format for example, the JATS will include both the TIFFs and lower-resolution WEBPs created to display the image online or in PDFs.

JATS export going forward

While Kotahi currently uses an on-demand model for creating JATS—where the XML is created and then destroyed after it’s downloaded—it’s possible that we’ll move to a model where JATS is created automatically every time a change is made in the production editor; this would happen server-side, and if the editor chose to download JATS, the file would already be ready. (In another model, creation of JATS might happen automatically when the user decides to publish the article, triggering creation of JATS, HTML, and PDF versions.)

A reason for this would be to make the JATS file available via Kotahi’s GraphQL server – where a web-based frontend might get files to show readers as HTML and offer PDF and JATS downloads.