If you've been around me for a while, you might have heard me go on and on about my long-term project: creating the ultimate document reader. As I try to design it, I am thinking of ways that the cognitive process of reading can be improved through computers and to guide my research, I have read a couple books about the neuroscience of reading.

One book in particular was quite invaluable: Maryanne Wolf's Proust and the Squid. I recently came across a few articles1 in which she discusses how she seems to have lost the ability to read and enjoy prose in long-form. She claims that this is because of how reading on screens has trained our brains to skim rather than use "deep reading" skills. Maryanne Wolf's next books will be on this specific topic and I look forward to reading them2.

Differences between reading on paper and on screens

Researchers in human-computer interaction, education, and psychology have long looked at the differences between reading on paper versus on screens (1). Getting to the root causes behind why there are differences between the two in terms of reading outcomes (e.g., speed, fatigue, comprehension) has been difficult because there really hasn't been a standardised methodology behind all the studies. Early studies found that some of the differences could be attributed to image quality and this seems to hold for single pages, but for longer documents, the ability to navigate and locate information is a more important factor.

Perhaps the difference can accounted for by our expectations of screen reading. When reading on a screen, we tend to read shorter texts such as e-mail or the news while doing other tasks such as checking notifications. This task switching may be occur many times a day as we navigate our computers. Some have suggested that the gap between these different modes of reading may be closed by adapting to screen reading in the same way that people had to adapt from reading on scrolls to reading from a codex.

A study of high-schoolers showed that those that were successful with tasks that required finding information online via a hypertext interface either had prior skills in linear reading or prior skills in basic computer navigation (2). This implies that a combination of those factors may be necessary to make the most of screen reading.

Another study compared students' performance with screen reading and paper reading when given two scenarios (3). In the first scenario, the students had a fixed time to read the material when given an assessment. In the second scenario, they were allowed to use as much time as they needed. What the authors found is that for the fixed time scenario, the screen readers performed on par with paper readers, but for the flexible time scenario, the paper readers outperformed the screen readers. The author's suggest that this difference can be accounted for by differences in how each group of readers self-regulated their reading time and that this difference in self-regulation comes from how readers perceive reading in each medium.

From these studies, I believe that two areas to improve when designing a document reader are document navigation and self-regulation of reading. There are many approaches to navigation, but I will not be covering that in this text. What I want to know is, how can we train ourselves to have better self-regulation?

I had a thought along these lines while I was studying a textbook. A major part of reading is going back to review what we just read. As we read, we must use our limited working memory to both visually process the words on the page and simultaneously pull out related concepts from long-term memory so that we can associate new ideas with our prior knowledge. In fluent reading, all this occurs in a short amount of time between reading chunks of text, so the number of associations that we can make depends on how quickly we can process all this information. In order to ease students into re-treading over what they just read, many textbooks use small prompts at the end of sections. These prompts are meant to help provide context for where the information fits into a larger structure and confirm that the reader understood what was written.

In an age where we have direct access to more information than ever before, many have turned to speed-reading techniques as a way to read more material — however, such approaches are not supported by eye-tracking research (4). Others have tried to replace reading entire books with reading summaries and while these give a decent summary of the arguments in a book, they are but tertiary sources — referencing a summary is not the same as referencing the book itself. Furthermore, if you only read the summary, you might not give yourself enough material to build a coherent model that you can recall later. Both forms of reading have their place, but if we want to improve our reading comprehension, it may be worth slowing down.

Metacognition

The activities found in textbooks are meant to aid what is known as metacognition — thinking about thinking. We apply metacognition whenever we take notes, break down a problem into subproblems, or set goals for studying. Metacognition is the process of recognising that we are thinking a certain way and applying a strategy to regulate and improve how we think.

The following are all valid metacognitive strategies:

• using a consistent system for note-taking,
• creating a sequence of tasks for working on a project,
• planning to read 3 sections in the next 30 minutes,
• doing all the exercises at the end of the chapter,
• devising mnemonics to remember a certain sequence of events,
• creating flash cards based on each new definition, and
• cramming 1 hour before a test.

These strategies might not be equally effective in all circumstances and for every person, but what they have in common is that they all regulate how we think. What I would like to investigate is can these metacognitive strategies be incorporated into the tools that we use to read, so that it is easier to apply them effectively and consistently?

The purpose of this text is to explore some thoughts on how to implement such a system.

Annotations

When I started thinking about metacognition, my first thought led me to textual annotations. Writing has long been suggested as a way to help improve memory. It may even be more important when dealing with electronic media because notetaking can be used to enhance the understanding of what we're reading. A recent study from 2013 suggests that the difference between paper and screen reading may be partially due to the distractions of multitasking; however, the effect of multitasking can be mitigated by using notetaking on paper as a way to retain focus (5).

Taking notes on paper is not a huge burden, but having those notes in digital form allows for more portability and, since not everyone has filing cabinets, makes it easier to retrieve years later. This ease of use comes with a tradeoff: digital notetaking may not be as effective for memory recall as taking notes by hand. In Mueller et al. (2014) (6), researchers looked at how students wrote notes and found that students that wrote notes by hand were able to perform better on assessments than students that wrote notes on laptops. This may be because there is more cognitive processing being done when choosing the salient points to write down from a lecture than when typing out the lecture verbatim.

How can this additional cognitive processing be elicited when keeping notes in digital form? One approach might be to use the Cornell method for notetaking. One way of using this method is to return to the notes after a day in order to summarise what was written. This additional summary promotes the synthesis and revision of the notes. When used correctly, the rewording required for summarisation provides the extra cognitive processing needed for learning.

Below is a simple mockup of how such a system might look like on a computer. Instead of showing a single page as in most document readers, this interface shows a page of the original document as well as another page where notes can be kept. The cue column is one the left of this page, the note-taking column is on the right, and the summary area is at the bottom of the page. When the notes for a given unit of the text (e.g., a section or paragraph) are complete, the time can be recorded so that when returning the next day, the user can be given a reminder to write a summary.

Mockup of Cornell notes UI

Flashcards

Another popular metacognitive strategy is to use flashcards. They make a good supplement to the Cornell method as the cues can be turned into the front of a flashcard and the notes can become the back. We can also create flashcards on their own and link them back to the text if further clarification of the notes are needed.

Mockup of flashcards UI

Just making the flashcards is not enough. They need to be reviewed. One way of reviewing called spaced repetition is based on creating an adaptive schedule so that difficult cards are seen more often than easy cards until the difficult cards have been memorised. There exist many techniques and tools for creating effective flashcards and scheduling the revision of cards, so those can easily be integrated into the document reader either by outputting data in the appropriate format or reimplementing the algorithms3.

Triaging and comparison

When researching a topic, using a single source of information may not be enough. I often go to the library and grab almost every book on a topic so that I can learn from many perspectives. As I read, I try to figure out the specific strengths of each document: some may cover more theory than others or one may have particularly informative diagrams. Being able to sort through each of these documents and decide which ones are relevant is possible on a computer through the use of folders and tagging, but this is sometimes very unsatisfying. For example, if I am reading papers for a literature review, I usually make a spreadsheet to organise details about what each paper is about. Before I actually commit to making a spreadsheet, I try to stack the papers that I have printed out into different piles based on quickly skimming the contents. This is known as document triaging.

One example of how I would use this is to separate a set of papers into the categories:

• Not relevant: papers that do not cover what I am trying to figure out in the current project (ignore these);
• Survey papers: papers that review the results of many papers at once (read through these and find any references I might have missed);
• Classic research: older research that may or may not be worth looking at (skim over these — might only be of historical interest);
• Recent research: more current papers (read these more closely).

One approach to a GUI for this might be to emulate the layout of a desktop area where different regions indicate different categories. This may be easier to visualise and work with than using different lists or drop-down menus as it uses larger GUI elements (c.f., Fitts's law). An example of what it might look like is shown below.

Mockup of document triaging UI

The workflow proceeds as follows:

1. The user chooses a document from the Queue on the right.
2. The chosen document appears in the center Reading area where the user can quickly look over the document.
3. Once a category for the document is determined, the user drags it to the appropriate category region on the left.

Perhaps this task can benefit from active learning which might be able to suggest categories for papers based on metadata such as page length or year of publication.

Another related task is being able to compare multiple documents side by side. Below is a picture of a bookstand modelled after a design that Thomas Jefferson created for his office4. It allows for keeping multiple books open at the same time and rotates so that switching between books is easy.

Picture of Jeffersonian revolving bookstand (taken from this project showcase)

In a way, this is like browser tabs. However, there are times when you may want to see two documents side-by-side rather than having to flip between them. For example, if I was reading two books on history written by different authors, I might want to see how both authors address the same topic. Whether or not this should be implemented as a multiple document interface (with tabbing and docking) is still not clear to me.

Reading strategies and serendipity

If you've ever read a scientific research paper, the first thing you'll notice is that they are highly structured: there are sections that are always in every paper (e.g., Abstract, Introduction, Methods, Results, Conclusion). These sections are meant to guide readers so that they don't have to read through the whole paper. There are certain strategies to reading a research paper5 which emphasise that you have to re-read the paper several times based on your goals (e.g., performing a literature review versus trying to reproduce the results). On each pass, you will try to answer different questions so you will want to spend more time reading specific sections. Perhaps these reading strategies can be turned into a checklist so that each paper has a progress bar that tells you how far along in understanding the paper you are. That way you can skim over several papers in one sitting and then slowly try to understand one paper at a time.

Sometimes the path you take with your reading does not follow a straightforward checklist. There are times when you are searching for something and you come across an unexpected connection which can lead to more creative thoughts. This can either occur when discovering new material or coming back to older material. Understanding how to create serendipitous encounters might be a little tougher than the previous metacognitive strategies. As noted in André et al. (2009) (7), serendipity is difficult to facilitate and study in a laboratory setting. One approach that I would like to try is to bring up older material that the user might have read months or years ago. Perhaps the older material might be seen in a different light now that the user is re-encountering it.

The system described by author Steven Berlin Johnson in this post titled Tool for Thought describes an interesting workflow where he captures quotes from books and uses a tool that can help find other related quotes in his library. This allows him to start with a single idea and then find other ideas that might be related to that original seed. One of the authors of (7), Susan Dumais, worked on a technique to do just this: latent semantic indexing. However, at this point, I am not certain how large of a personal library is needed for the gains from this technique to become apparent.

Conclusion

As far as I can tell, none of these techniques exist within a single existing application. Perhaps that is because using such a complex application would become daunting — there would simply be too many features. Furthermore, since each of these metacognitive strategies can be applied in many different ways, another challenge will be creating tutorials that show how to use them effectively.

I'd really appreciate any feedback on these ideas. They will certainly need tweaking before they become usable.