1 Department of Computing, University of Bradford, School of Computing, Informatics and Media, Bradford, West Yorkshire, BD7 1DP, UK;
2 De Montfort University, Head of Department of Media Technology, Leicester, LE1 9BH, UK
The relationship between students learning traditionally or through virtual worlds using computer-oriented tools is of keen interest. Although, the extent to electronic learning varies in great degrees from entire online environments to partial complimentary tools which differ according to their functions. The focus of this research paper is to discuss the paradigm shift from traditional means of study to computerisation, in particular relating to the area of note-taking. Research into cognitive factors associated with learning and performance including memory have put forward suggestions, which could assist the cumulative learning process. Comparative analysis between a number of note-taking techniques refined the study with the electronic adaptation of the popular Cornell method with the proposed En-AISR platform. Emphasis has been placed on the influence and significance towards the amalgamation of multi-modal features to enhance and stimulate students learning experience. A two-group study measured students learning, performance, and experience between both systems using usability criteria. Outcomes from this experiment suggest a positive influence of a multi-modal note-taking tool as a complimentary learning aid
open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.
An Empirical Two-Group Study into Electronic Note-Taking
In all walks of life obtaining knowledge is customary and can be captured in a number of ways. This can be achieved through deliberate means such as picking up a book or passive by listening to someone. The content of knowledge is translated by individuals and acquired by the skill of note-taking. This skill differs from person-to-person and is dependent upon personal preference of the technique utilised. The core entity associated with note-taking is students, who on a regular basis are required to undertake this skill on a mass scale. The processes involved in this task are studied including the encoding, storage and reviewing process. Research into various note-taking techniques including, Cornell, Outlining and the Mapping method are discussed resulting in the computerized adaptation of the Cornell method. Research into cognition and multi-modal stimuli factors present an understanding regarding student perceptions, attitudes and behaviours. An analysis into the factors contributing to designing an e-learning environment with underlying pedagogical principles, suggests the usefulness of computerisation. Furthermore, the introductions of Tablet PC’s and software applications comparable to Microsoft OneNote are studied.
Formation of a note-taking technique had been undertaken based on previous literary works, rigorous testing and numerous studies. The result, En-AISR (Electronic Notes Application for Input, Storage and Retrieval) note-taking platform, was then comparably tested with an electronic adaptation of the Cornell note-taking method. The two-group study between both computerised systems demonstrated the usefulness of computer-oriented learning and multi-modality. Furthermore, student perceptions towards a computerized note-taking tool are learned alongside factors influencing usability issues.
This paper is divided into six main sections. The first part provides an insight into relevant literary works associated with note-taking, note-taking techniques, cognition and multimedia, and e-learning. The second section introduces the electronic platforms. The third part presents the design process and procedure for the two-group study defining the variables, hypothesis and usability measures. This is followed by results obtained from the experiment including statistical data. Penultimately, the discussion critically analyses the results in reflection of the literature and finally, the paper concludes by stating the importance of note-taking as a facilitative tool within education and future research.
As this is an initial study aimed at determining feasibility of a note-taking technique as an electronic tool, it will set the foundation to encourage the incorporation into a virtual environment. The findings will report suitability of a technique, availability of electronic technologies, and multi-modal associations that assist in the formation of a virtual environment, alongside the psychological impact of such senses relating to this research area.
Theoretically, note-taking is perceived as the transfer of information from one mind to the other typically, communication delivered by speech and assisted with anecdotes. The practical reasons for taking notes are to use them as helpful tools when reviewing and for the assistance they provide to learning material [1-3]. During the encoding phase without reviewing, note-taking does facilitate learning. However, this is affected by the quality of the notes. There is a 34% chance of recalling textual information if it is noted but a 5% chance if it is absent . Additionally, if notes are made but not reviewed this is still positively shown in test performance . This finding suggests note-taking aids but does not ensure recall, which is encouraged by the reviewing process . Ideally, reviewing should be carried out nearer the exam to achieve maximum value . Many studies have found students capture less than 40% of the lecture content [4, 8] typically, around 20-40% of the important ideas . To improve this, students must be given training as they actively participate before, during and after lecture. Thus, the elements of what needs to be captured are influenced by their purpose, interpretation and technique .
To date the post popular note-taking technique is the Cornell note-taking method, also referred as ‘Do-it-right-in-the-first-place’. This method was developed by Professor Pauk at the Cornell University over forty years ago. The purpose of the method is to arrange and condense notes in a systematic manner without having to do multiple recopying . The method has three specific areas (Fig. 1); Area A assigned to keywords, Area B, to take notes in and Area C as a Summary.
Area A is assigned to keywords or phrases after lecture  however, over the years an alternative has been questions, aiding recall over recognition. These cues are known to assist memory and act as a reminder as well as helping to identify relationships, also referred to as the Q-System . Area B, remains for the recording of notes during lecture. The student tries to capture as much information as possible here. Area C, is left to summarise the notes, allowing students to reflect upon the main ideas of the lecture .
The main advantage of this technique is its straightforward and organised structure to capturing notes. This technique is also suitable for technical modules such as Mathematics and Physics and non-technical subjects such as English, and History. During an engineering and applied sciences workshop, experiment involving 70 students, participants stated this technique is “simple to do” . The authors  suggested the technique “provides organization, requires interaction and concentration” and so a scheduled review can be carried out straight away highlighting keywords. Furthermore, students are able to summarise content which “improves comprehension through individual understanding” .
In comparison, the Outlining method (Fig.. 2) has a more meaningful spatial organisational layout, implicitly encoding conceptual relations . The method consists of dashes or indentation and is not suitable for technical subjects such as mathematics or physics. This technique requires indentation with spaces towards the far right for specific facts. Relationships are represented through indentation. The main advantage of this technique is the neatly organised structure allowing reviewing to be carried out with ease. However, the outlining method requires the students full concentration in order to have optimum organisation of notes therefore, the technique is not ideal if the lecturer is going at a fast pace. The method has been disapproved because of its organisation structure being confusing . This is mainly due to the arrangements of numerals, capitalised letters and so forth.
Example of the outlining method.
In contrast to the Cornell and Outlining method, the Mapping method (Fig. 3) is a graphical representation of the lecture content. Students are stimulated to visually determine links illustrating relationships between facts and concepts . Concept maps enable brainstorming, break down and representation of complex scenarios, identifying and providing solutions for flaws and summarising of information. To enhance accuracy students must participate actively and initiate critical thinking. The disadvantage of this method includes lack of hearing changes from points to facts  and great concentration skills are required.
Example of the mapping method.
Cognition and Multimedia
Within cognition the processes associated with note-taking include attention, span, tasks in working memory, encoding and exploring long-term memory. These processes help to make a distinction between weak and effective learners. These cognitive processes can be best described using a “discourse processing model” . The model assumes and envisages comprehension at three levels of strategic complexity; propositional, local coherence and macrostrategic. Meaningful and complex propositions from within a paragraph that convey a single idea are processed at the propositional level. Combining the complex propositions and identifying the relationships is a part of the local coherence level. The outcomes ‘macropropositions’ are then amalgamated with macrostrategies to form an understanding of the subject area. In theory, during note-taking all three levels are involved because the note-taker combines the lecture content to establish meaningful macropropositions .
Presently in higher education, lectures are still known as the most common mode of instruction where students attend and record notes based upon the lecture content . Additionally, lectures are known as a prime source of student information, engaging their interest within the topic. It has been learned that students take notes in every lecture regardless of their reason and the motive for this is because of their usefulness to learning and due to social pressure .
Whilst reading or listening the human mind has a tendency to wander off thinking about thoughts other than what is being taught. Student learning is assisted by note-taking enhancing attention span and enabling the organisation and remembrance of the topic. During learning, effects are known to occur within the encoding and storage phase. The encoding stage involves students capturing relevant content and the storage phase reviewing the notes. To ensure maximum benefit, the encoding and storage phase should be combined together .
Strategic note-taking, identifying cues, organising information and combining prior knowledge to existing knowledge improves comprehension. Moreover, rehearsing is a popular strategy but weak as it does not transfer information into the long-term memory. Therefore, students should attempt to understand and relate learned content. Variations in cognition, within the working memory control processes influences note-taking, this is because information is held and manipulated there. Findings suggest students with low memory ability record smaller number of words, complete ideas . To enhance achievement note-taking can be combined to study content, forming questions, this has been positively demonstrated .
Within learning and teaching, the exploration of multimedia is an upcoming issue. Information can be represented by visual, aural, haptic and other senses. Non-speech audio incorporated within interfaces is becoming increasingly popular. When used alongside visual output it increases the amount of information communicated and reduces the amount received through the visual channel. Sound is more flexible, heard at 360° compared to visual, where the retina subtends an angle of 2° around the point of fixation. Therefore, sound is an effective means of capturing user’s attention. System earcons can be used to convey sound. These comprise of musical instruments in a short rhythmic sequence with varied intensity and timbre .
E-learning, also often referred as distance education utilises a number of technological devices. Educational institutes are today known to deliver academia over the internet. The internet has great potential, allowing not only learning material to be taught but also for collaborative learning to take place. Within the next ten years the growth of online student learners is predicted to reach 5 million from 240,000 .
To have a successful e-learning system all sub-components and interrelated processes must be considered. This is because if one process fails the entire system can fail  therefore; underlying pedagogical principles must be derived . These include considering the user’s behaviour towards the system as it is an isolated activity and users can become frustrated. Also, as the internet has a vast amount of knowledge it can be presented in a bias manner providing users with partial information. Considerations for the environment and the user actions to be performed to achieve a specific goal must be clearly outlined. Furthermore, user’s interpersonal skills including their attitudes, perceptions and behaviour are central to affecting the effectiveness. E-learning reduces teaching time, increases proficiency and improves retention , nevertheless, this is not always true as one particular study presented lecture notes online and results showed students performed weaker .
E-learning is provided globally, allowing users to read books online, annotate, and collaborate by discussing subject content. Research has shown the use of an online notepad can achieve higher then pen and paper methods . Reinforcing this point, annotation increases efficiency in a number of ways including supporting memorisation , improving comprehension , encouraging critical thinking  and allowing clearer understanding of text . Annotation applications introduced include, Microsoft Word and OneNote that concentrate on annotation and Sharepoint™, these allow manipulation, editing and annotation simultaneously.
Many institutes have integrated Tablet PC’s [32, 33] as a medium to replace the blackboard. These are typically connected to a data projector so students are able to make notes that are visible to the rest of the class. During an experiment utilising these results demonstrated students had a better understanding of the lecture and concentrated more . The use of Tablet PC’s is known to increase and enhance a greater collaborative learning environment  with increased interactivity being its main benefit .
The significant difference between the traditional manner of learning compared to computerised is the “medium over which it is transmitted” . The flexibility enables students to learn at a time and place of their choice however, this in term of feasibility, whether learning should be entirely web-based is arguably one of the most important factors [37, 38]. The major difficulties faced by the e-learning shift are the drive for motivation and culture clash. Many learners are just not prepared to accept the change and so prefer the traditional means of study .
Two electronic note-taking systems have been developed with specific consideration into the “eight golden rules of interface design” outlined by Schneiderman . The first designed computerised platform is an adaptation of the Cornell note-taking method (see Fig. 1), resulting in the E-Cornell, Electronic-Cornell. The second serves as the experimental platform, En-AISR (Electronic notes-Application for Input, Storage and Retrieval). The note-taking method consists of three main components; a Que column, a primary note-taking area known as N1, and a secondary note-taking area known as N2.
Purpose of the Que column is to enable students to form cues, keywords or markers for the notes captured in the note-taking area. These cues can be made up of a single word or phrase. From these reminders, students fall within the process of recognition and therefore these cues are able to assist recall by memory ‘retrieval’. N1 and N2 serve as annotation areas whereby students are engaged in the encoding process and the ‘input’ of notes. The primary note-taking area, N1 is a blank area of white space designated for the initial recording of notes that can be textual and/or graphical. N2 however, is composed of fine lines to form a regular grid enabling students to draw graphs, write formulas or expressions and draw diagrams.
Originality of the initial structure of the traditional Cornell note-taking system was maintained. The design and implementation of both platforms were consistent in terms of the processes of note-taking and operating instructions. The multi-modal interface design was largely inspired by the need for a more flexible, effective and efficient tool supporting human computer interaction. The experimental platform consisted of multimedia metaphors including, visual, aural and speech multi-modality. Hence En-AISR is distinguished by method and multi-modal features.
EXPERIMENT DESIGN PROCESS
Usability parameters have been used during the two-group study. These can be measured, providing a measure of acceptance practically and socially . The ISO standard defines usability as the “[e]xtent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction” . Furthermore, effectiveness is the “[a]ccuracy and completeness with which users achieve specified goals”; efficiency is the “[r]esources expended in relation to the accuracy and completeness with which users achieve goals”; and satisfaction is the “[f]reedom from discomfort, and positive attitudes towards the user of the product” . Comparatively, recommendations into other usability measure include 1) time to learn; 2) speed of performance; 3) users rate of error, 4) retention over time, and 5) subjective satisfaction . Similarly, Nielsen  recommends measuring 1) learnability, 2) efficiency, c) memorability, 4) errors, and 5) satisfaction. During this experiment the parameters being measured will include effectiveness in place of learnability, efficiency, error and satisfaction.
A 24 participant sample had been selected randomly to test both systems on a rotation basis. Four case studies had been assigned to each participant and system thus; the random rotation of subjects and case studies was to sustain the learning effect. This study undertaken by 24 participants was grouped into four groups. Each group had six participants.
It was learned that a substantial 75% of the sample are note-takers of which 33% take notes in every lecture and 42% occasionally.
The aim of this study is to test the following hypotheses:
H1: En-AISR will have a positive correlation of correct to incorrect answers, in effect reducing the error rate/latency in performance tests.
H2: The incorporation of multi-modality within En-AISR will improve the learning experience and will be preferred.
Throughout the design process it is essentially important to define variables.. These can be of three types; independent, dependent and controlled. Maintaining consistency throughout the experiments ensured these. Independent variables in the experiments were both note-taking systems; E-Cornell and the En-AISR. Dependent variables consisted of accuracy in answering tasks therefore, measuring the percentage of tasks completed successfully and error rate; task completion time excluding the time taken to read or prepare; and users satisfaction response to the systems on a 5-point Likert Scale.
A short training session was provided to all participants demonstrating how to operate the systems. This followed with providing subjects with a transcript of the case study. These case studies had been recorded on an audio file. The audio file was played once only and subjects recorded as many notes as they felt necessary about the case study using the assigned note-taking system. Subjects could follow the transcript or refer to it at any time. Once the audio recording had finished participants were given a 5-minute review period. During this period subjects could only use the transcript to add or edit their personal notes. After the review period the transcript was taken away and using their own notes subjects had to answer 12 questions, six recognition and recall questions. Each recognition question had four options. The time taken to answer each question was measured in milliseconds (ms). At the end of testing each system, a subject satisfaction questionnaire was completed. The System Usability Scale (SUS) had been used consisting of ten statements. Additional statements had been incorporated depending on the system.
A breakdown of results observed during the two-group study has been documented besides the relevant parameter being measured.
The total number of correct answers found E-Cornell with a total effectiveness rate of 77.78% in comparison to 91.67% attained on the En-AISR. Fig. (4) depicts the number of correct answers achieved by each user on both electronic platforms. Subjects using the E-Cornell observed an average of 9.33 correct answers with a distribution range of 7.67 – 10.99 around the mean, hence σ1.66. En-AISR noted an average of 11 correct answers per subject with a lower variability of σ1.29.
A comparison of correct answers achieved.
Wilcoxon Signed Ranks Test calculates 18 views in support of H1 and four counter-evidences. On average this derives a positive split of 18/4. The mean rank value of positives is 12.44 against an average of 7.25 for negatives. The sum of ranks is positives 224.00 to negatives 29.00. Therefore, there are a greater number of positive ranks with a larger value. Wilcoxon Signed Ranks Test derives a Z-Value of –3.201 with significance of p = 0.001. As a result, the En-AISR note-taking system outperformed the Cornell with a higher rate of effectiveness.
This measured the time taken to answer all questions using the electronic platforms, illustrated in Fig. (5). The total time taken on the E-Cornell was 3443700.00ms. The average time taken for each question was 11957.29ms with an average calculated time of 143487.50ms per subject for all 12 questions. In comparison, En-AISR observed a total time of 2199320.00ms. The average time calculated per question was 7636.53ms and the time calculated per subject was 91638.33ms. Statistical significance between both systems in terms of performance finds E-Cornell with a larger mean value than En-AISR, 143487.50ms and 91638.33ms. The difference in mean values is equal to 51849.17ms. Using paired samples T-Test this derives a t-value of 5.772, 23df, p = 0.001. As a result accepting H1 as subjects using En-AISR completed tasks more efficiently.
A Comparison of time taken to answer questions.
Time Taken to Answer Questions
E-Cornell, reports a total of 64 incorrect answers calculated as 22.22% with an excess time of 2.61%. The average number of incorrect answers was 2.67 per subject with a dispersion of σ1.66. In contrast En-AISR noted a total of 24 incorrect answers deriving a percentage of 8.33%; almost three times less than E-Cornell. The average per subject was one incorrect answer with a dispersion of σ1.29. The total excess time computed for this system was 0.86%. A large difference was observed between both systems, of 40 incorrect answers with subjects using En-AISR performing significantly better. The excess time was three times greater in E-Cornell than En-AISR. Statistical analysis using Wilcoxon Signed Ranks Tests based on the rate of error finds incorrect answers between E-Cornell and En-AISR derives a p-value of 0.001. Results between excess times using Paired Samples T-Tests computes a larger mean for subjects using E-Cornell then En-AISR -3739.83ms as opposed to -786.97ms. The difference in averages is 2952.85ms and dispersion σ2978.50ms. Thus, t = 4.857, 23df, p = 0.001. Based on these results, the En-AISR note-taking system had a reduced number of incorrect answers and excess time and so the null hypothesis is rejected.
SUS Scores from subjects rating the E-Cornell found an average of 53.96% whereas En-AISR observed a higher average of 78.75%. The spread around the E-Cornell average was almost double than the En-AISR, σ26.04% as opposed to σ14.37. Descriptive statistics present a range of 82.50% on the E-Cornell and 50.00% on the En-AISR. Inter-quartile ranges were 46.87% in the E-Cornell whereas; En-AISR noted a closer inter-quartile range of 23.13%. A comparison of scores attained on both systems has been illustrated in Fig. (6). On the whole, statistical significance between the two systems comparing average SUS Scores using Wilcoxon Signed Ranks Tests finds 19 positive ranks in favour of H1 and five counter-evidences. The positive mean rank is 13.79, a sum of rank 262.00 whereas the negative mean rank is 7.60, sum of rank 38.00. As a result, a larger proportion of positives are noted with a higher mean value deriving Z = 3.201 with a significance of p = 0.001. This determines subjects using the En-AISR note-taking platform had a more satisfying experience than with the E-Cornell, accepting the test hypothesis as true.
A Comparison of sus scores
With regards to the methodology of the note-taking methods, responses towards the statement I feel the cue column is irrelevant derived Z = 1.844, p = 0.0325. This verifies the relevance of the cue column is best suited in the En-AISR than the E-Cornell. E-Cornell subjects were in agreement finding the summary area unnecessary however; the usefulness of the graph area in the En-AISR was highly rated at four on the Likert Scale.
Additionally, subject feedback towards interactivity demonstrated the En-AISR system has a higher level of interactivity and provides greater comfort with a z-value = 2.955, p = 0.0015. Subjects using En-AISR found the visual multi-modal metaphor more useful then its usage on the E-Cornell, hence higher value towards multi-modal capabilities incorporated in the system. Statistics compute Z = 2.009, p = 0.0225 towards this statement.
Having researched the various note-taking techniques, the study has demonstrated the effectiveness of the Cornell note-taking method. This is in terms of simplicity, usefulness and popularity, in comparison to the Outlining and Mapping method. Previous studies conducted on these methods [11-16] and regarding cognitive learning parameters [1, 17-19] have reinforced the aforementioned points and additionally, the appropriateness and ability to deploy this method within any subject area provides a stable foundation to build upon.
As note-taking is a broad activity amongst students conveyed typically by speech and assisted by anecdotes, the processes of encoding and storage are fundamental. The importance of communicating information from one mind to the other and rehearsing these ideas is clearly reflected in student performance and so the combination of both processes is vital. Therefore, to facilitate learning, recall must be enhanced and attention to detail must be encouraged to devise more concise lecture notes.
E-learning environments vary from providing a complete academic learning system to either computerising a sub-component. The flexibility to learn anytime, anywhere is desirable however, as it is an independent experience, it is essentially important to consider the need for a collaborative structure with the exchange and interaction of knowledge
Findings from the comparison between both electronic systems, E-Cornell and En-AISR concludes the En-AISR note-taking system outperformed the E-Cornell in terms of effectiveness, efficiency, reduced rate of error and subjective satisfaction. Statistics evidently proved a clearer subject understanding of the En-AISR with a higher measure of achievement. An outstanding total number of correct answers were obtained by subjects using En-AISR by 91.67% as opposed to 77.78% when using E-Cornell. Excess time on the En-AISR was three times less than the E-Cornell showing redundancy in time consumed and efficiency in performance. Support towards the suitability of multi-modality was proved and the interactive experience with the En-AISR was most favoured.
In conclusion, the study attempted to examine electronic note-taking. This was undertaken by a comparative analysis of techniques identifying processes involved and psychological cognitive factors. It has been learned that the Cornell note-taking technique is appropriate and effective. Furthermore, the introduction of a learning environment provides weight for the possibility of computerising a sub-component of learning, the En-AISR note-taking technique. Outcome of the two-group study towards usability parameters demonstrate positive results towards the computerisation of an e-note-taking tool, in particular in favour of En-AISR and allows scope for further rigorous testing. Therefore, as the basis of the methodology has been accepted alongside the amalgamated components on the En-AISR, future work will specifically involve experiments testing multi-modality and incorporating this within a virtual environment.
= Electronic Cornell note-taking method
= Electronic Notes Application for Input, Stor- age and Retrieval
= Human Computer Interaction
= Primary note-taking area in the En-AISR methods
= Secondary note-taking area in the En-AISR methods