What verb does 'negotiation' go with? Do you 'make' or 'conduct' a negotiation? Adam Kilgarriff, Director of Lexical Computing, explains how 'corpora' can help us answer such questions where dictionaries meet their limits. He presented a British Council seminar on the subject yesterday.
What is a corpus and how does it differ from a dictionary?
A corpus is a collection of texts. We call it a corpus (plural: corpora) when we use it for language research.
That makes your class's essays a corpus - a small one. It also makes the internet a corpus - a big one.
People writing dictionaries are in the vanguard of corpus linguistics. If you are writing a dictionary, the biggest crime is to miss things: to miss words, to miss phrases or idioms, to miss meanings of words. Lexicographers (the people who write dictionaries) have known for a long time that the best way to avoid missing things is to have a big corpus, and a computer. The computer can then find all the words (ordered by frequency) so a lexicographer can check the list to make sure that words are not missed.
Words in context: Finding out how words are used in a language
It can also show them all the examples of a word in context. This is called a concordance. By running their eye over the concordance, lexicographers can find all the meanings of the word, and phrases it is in.
If it is a big corpus, or a common word (or both), there might be thousands of examples of the word. Then, the computer can go one step further, and prepare a 'word sketch', a summary of the contexts, collocations and phraseology for the word.
This is how contemporary lexicography works. Lexicographers start from the word sketch, which gives them a good idea of what they must not miss. They then work out what different meanings, grammar and phraseology are shown by the collocations in the word sketch, and write definitions for them. They can also use the corpus as a source for example sentences.
When I say 'the computer', of course I mean an app that indexes the corpus and lets users make concordances and word sketches. Google is one app that does something like that (with the Internet as its corpus). However it is not designed for people doing language research. One that is widely used for making dictionaries, with lots of corpora in it, that made the screenshots above, is the Sketch Engine.
Dictionary-makers were leaders in corpus use. Following on were people writing language courses. They wanted to make sure that the facts they were teaching about the language were in fact true (!), and to teach common patterns before rare ones, and to use authentic examples of the patterns.
Should teachers use corpora?
So, in English language teaching, there is plenty of indirect corpus use, via dictionaries and course books. What about direct corpus use, by teachers, even students? Should you use corpora?
My answer is: yes - if the dictionary does not tell you enough. If you want to find out what 'negotiation' or 'secede' means, you could start from a corpus but it will be long and slow: better to look it up in your favourite dictionary. But if you know 'negotiation' and want to use it, but are not sure what verb to use it with, then the leading learners' dictionaries give little help. The word sketch, on the other hand, promptly shows you that people resume, (re)start, (re)open and conduct negotiations, and that negotiations stall, fail, get bogged down, drag on and even collapse (each item can be clicked, to see examples of the collocation in use.
A second situation is the teacher marking work, whose English is good but who is not a native speaker. A student's essay has 'seceding out of Ukraine' - is that OK? A quick check of the concordance for 'secede' shows that a region secedes from a country.
Another consideration is always student motivation. If a class is currently engaged with volcanoes, it would be nice for them to look at the English of volcanoes (I've felt an affinity for volcanoes ever since my big end-of-primary-school project). So can we have a volcanic corpus? Yes! The Sketch Engine has an instant corpus tool, where text on a topic is gathered from the web in a few minutes (by a teacher or, as a class exercise, by the students) and this is then the data for a mini research project.
You might also be interested in:
Read more articles
Writing a Paper in English Corpus Linguistics1
By Stig Johansson and Hilde Hasselgård
Choosing a topic, Using sources, The investigation process, Writing strategies, Some characteristics of linguistic discourse, Supporting your statements, Organizing the paper, Language use, Links
A term paper ('semesteroppgave') is an exercise in linguistic method. The object is to show that you can:
- define a research question
- account for your choice of theory and method
- select and analyse a limited material
- conduct a theoretical discussion of linguistic problems in English
- use secondary sources in a critical way
If you choose your topic wisely, you may also make a contribution to English linguistics.
2 Choosing a topic
Your teacher may suggest possible topics that relate to the course you are taking. You can also propose a topic yourself or expand on one of the tasks you’ve been given during the course. Another possibility is to carry out a study similar to one that you have read about in the course syllabus. It is important to be realistic and limit the topic, so that you can finish your paper within a reasonable period of time and write it up within the stated 10 standard pages. Contact your teacher at an early stage to draw up a plan for your paper. The topic will often have to be restricted and modified as you go along.
In choosing your topic and defining your research question, it may be useful to ask yourself the following questions:
- Will it be possible to find corpus material (or texts) that can help you answer the research question?
- Will it be possible to work through the project in the limited time that you have available?
- Do you know what else has been written on the topic?
More about choosing a topic and defining a research question.
Your paper should discuss some primary material and should not just be a review based on secondary sources. Primary material is the actual linguistic data you write about, such as written texts, transcriptions of spoken material, tape recordings, or elicited responses from native speakers. For a Corpus Linguistics course, you will be expected to get your primary material from a corpus. Secondary material is what has been written previously on the topic.
3.1 Secondary material
Start by making a general survey of secondary material. This will show you what has been done before (which means that you do not have to do it) and will probably give you ideas on how you should (or should not) deal with your primary material, on problems that you had not thought of, etc. You can find secondary material in a number of places, such as:
In connection with a term paper there is of course a limit to the amount of secondary sources you can be expected to go through. Consult your teacher, who may advise you on this point.
In going through the secondary material, you should make notes and collect excerpts as you go along. It is important to organize your notes in such a way that you can survey (and rearrange) them while you are working on your paper. Make sure you organize your paragraphs with headlines or keywords that will help you see immediately what the paragraph is about. If you write on paper, write on one side of the paper, to make it possible to reorder the notes or add to them. Make sure that you copy the excerpts correctly and write the name of the source and the page number immediately. This will save you a lot of extra work and trouble later on. Getting the facts right is fundamental in scholarly procedure.
3.2 Primary material
Though it is important to study the secondary material carefully, your main task is to collect and analyse some primary material. You may draw your primary material from corpora such as the LOB Corpus, the British National Corpus (BNC), the Corpus of Contemporary American English (COCA), the English-Norwegian Parallel Corpus (ENPC), the Oslo Multilingual Corpus (OMC), the International Corpus of English (ICE), or the International Corpus of Learner English (ICLE, including the Norwegian subcorpus NICLE). The corpus you choose depends on your research question. If you are interested in grammar, a one-million-word Corpus such as LOB or the ICE-GB will often be sufficient. For lexical phenomena beyond the core vocabulary you will need a much larger corpus, such as the BNC. If you want to study translation or compare languages, you will find a good source in the ENPC or the Oslo Multilingual Corpus (OMC). Aspects of learner English can be studied on the basis of the International Corpus of Learner English (ICLE) or VESPA.
Find out more about available corpora
To use a corpus, you need a search programme. Many corpora can be searched by means of a general concordancer such as AntConc or WordSmith Tools (see Scott 2001), but for example, the BNC, ICE-GB (the British part of ICE), and the ENPC have their own search engines.
It is often useful to make a pilot study first, i.e. to collect a small amount of material and analyse it along the lines intended. This will show you if your material is giving you enough data, and the right kind of data, or whether you may have to use different kinds of material.
Be careful in registering your material. Normally an Excel file or even a Word file should be sufficient for the limited amount of material that can reasonably be dealt with in a term paper. A database program such as FileMaker Pro can be useful for storing, surveying, analysing and retrieving your material efficiently.
Whichever system you use, make sure to copy each example correctly, to copy enough context, and to note the source correctly at once, to save time and trouble later on. This is because you may want to look up the example in the corpus again (for example, if you need to look at the wider context) and because you should specify the source when you give quotations from your material.
4 The investigation process
We can broadly distinguish between the following stages:
- data collection
4.1 The research question
Suppose you are interested in studying the position of the direct object with phrasal verbs; i.e. do you say she switched the light on or she switched on the light? The secondary sources will tell you that a pronoun as direct object is placed between the verb and the particle (e.g. she switched it on), while a noun or noun-headed phrase may appear either before or after the particle. You decide to focus on the word order problem in the latter case. The working title of your term paper could be "What factors determine the placement of nouns and noun phrases as direct objects of phrasal verbs?".
4.2 Data collection
To start your investigation you first need to collect relevant examples from a corpus. The examples may be extracted in the form of concordance lines, sentences, or other units, depending on your research question. Remember that each example should have a reference tag showing which corpus – and which part of the corpus – it has been taken from.
In collecting the material you immediately meet a number of problems. It is not always easy to distinguish phrasal verbs (e.g. she put on her coat) from other superficially similar constructions, e.g. constructions with prepositional verbs (e.g. she called on Mr. X to speak) or with prepositional phrases as adverbials (e.g. she called on Monday). And what about examples like these (from the LOB Corpus): Dr Horn swayed two or three inches back, Ugo had his glasses off now, I own some land up in the foothills? The first example looks superficially like a construction with a direct object, but should no doubt be analysed as containing an adverbial. In the last example up clearly goes with the following prepositional phrase rather than with own. The second example is a real problem. Do we recognize have off as a phrasal verb?
Collecting the primary material is not easy. It is necessary to be alert, so as not to miss out relevant examples or include irrelevant ones. If you have studied the secondary material carefully, you will have a good idea about what to look for and how to distinguish relevant from irrelevant examples. Nevertheless, there will always be doubtful examples. Make sure to include these, with a note on the type of problem. Such material is usually important to discuss in your paper.
As you go on you may discover that it is necessary to limit the material a great deal. For example, in a study of phrasal verbs, it may be necessary to limit the search to certain verbs with different particles, or to certain particles with different verbs. A dictionary of phrasal verbs will guide you in the process.
In a study of phrasal verbs (1986) Stig Johansson decided to focus on all examples in the LOB Corpus of six lexical verbs co-occurring with a noun or noun-headed phrase as direct object and ten particles; see Table 1. To find these, it was necessary to search for all forms of the relevant verbs followed within a certain span by one of the ten particles. Such searches can be easily done using a program like WordSmith Tools.
After irrelevant instances had been discarded, the material was as presented in Table 1. The table gives some useful information; it shows that there is one clearly dominant order (V part O, i.e. verb + particle + direct object). But this is only the starting-point of the analysis.
4.3 Classification of the material
The next logical step in the investigation process is the classification of the material. It is natural to start thinking about the classification while the material is being collected. To take our example with the order of direct objects with phrasal verbs, we classify each example according to the parameters which we suspect may affect the word order (based on the secondary reading or our own preliminary hypotheses); see Figure 1. Here FileMaker (or a similar database program) is an invaluable tool. Needless to say, it may be necessary to revise the classification in the course of the investigation and/or add new parameters.
4.4 The analysis
Having classified the examples, we can now rearrange and analyse the material in various ways. In a quantitative study it is usually best to draw up tables first, and then go on to describing and commenting on the data. Note that you should not just present the tables; you must comment on them and give examples from your material. Make sure that each table is numbered and has a legend that says what the table is supposed to show and explains any abbreviations or codes. Be careful in using numbers. Do not give percentages without presenting the raw frequencies. If you give averages, also provide some measure of dispersion. In addition to tables, you may wish to present figures or diagrams. These too should have a legend and adequate explanation.
4.5 Discussing the findings
To continue with our example case, see Tables 2-5. These show that the less common order (V O part, with the direct object between the verb and the particle) is more frequent with a short direct object, a definite form as direct object, and with a literal meaning of the verb plus particle combination. We can now deal with each parameter in turn, illustrating the main tendencies and commenting on any deviations from the main tendencies. We find, for example, that the four examples in Table 2 of V O part with a direct object consisting of three or more words are all definite and that the particle in three of the examples is followed by a prepositional phrase indicating direction, as in:
(1) ... bring these opulent days back to life ... LOB F35:68
With respect to the seven examples in Table 3 of V O part with indefinite noun phrases, we find that the direct object is short (one to two words) and in several cases followed by a prepositional phrase indicating direction, as in:
(2) The barman put two glasses down on the counter. LOB L11:73
With both literal and figurative meaning (Table 4) there is a clear preference for V part O, but the preference is much stronger in the latter case. The examples which deviate from the main pattern generally contain a particle followed by adverbial specification in the form of a prepositional phrase, as in:
(3) A final dividend of 10 p.c. brings the total distribution up to 17.5 p.c. ... LOB A16:178
Such adverbial specification is also common in the V O part pattern with combinations used in a literal sense.
Advice on how to quote examples in your paper
Analysing the material means asking interesting questions about it. In our case we found three parameters which turned out to be important: the length of the direct object, the definiteness of the direct object, and literal vs. figurative meaning of the verb + particle combination. The minority pattern V O part is used particularly with short and definite noun phrases as direct object and when the verb + particle combination has a literal meaning. In the process of the analysis we discovered another important factor: the occurrence of adverbial specification after the particle in the form of a prepositional phrase. This strengthens the minority pattern V O part.
The results may not always be clear-cut; see Table 5. There seems to be a slight tendency for the minority pattern V O part to be more frequent in fiction. In such cases we need to bring in statistical tests. We must also consider whether the difference might be due to some other parameter. Could it, for example, be a reflection of the length of the direct objects? We could expect noun phrases as direct objects, like noun phrases in general, to be less complex in fiction than in informative prose. If there really is a difference between fiction and informative prose which is not due to some other parameter, what might it be due to? These are the sorts of questions that must be asked.
If our results are to be of real value, we should try to generalize beyond our data and find deeper explanations for the regularities observed. Do we have reason to believe that the regularities we have noted extend to phrasal verbs in general (and do not just apply to the six verbs and the ten particles we selected for our material)? Can we find a deeper explanation for the regularities observed? In our case we can relate two of our parameters to two general word order principles: end weight (cf. Table 2) and end focus (cf. Table 3; indefinite noun phrases are more likely to introduce new information and appear in final focus position). It seems reasonable to suppose that a figurative combination (cf. Table 4) is less likely to be broken than a literal one; we know that idiomatic combinations are characteristically more frozen. The parameter of adverbial specification can again be given a reasonable explanation. Note that there is a tendency for adverbial particles to be attracted to related prepositional phrases; the result may even be a compound preposition: into, up to, out of, etc. An indirect result of this attraction is that we get the minority pattern V O part, with the particle plus prepositional phrase in final focus position.
In other words, we have answered the question posed at the start of the study and have been able to relate our findings to other phenomena in the language. We have reached the final stage of the investigation process and can finish the writing of the paper.
5 Writing strategies
The writing of the paper should not be deferred to the last day before the deadline. It is natural to write notes while working on the material, such as comments on examples or notes or ideas that you think may become useful later. You might even have a brainstorming session at an early stage when you write down in telegraphic form whatever you think is relevant to the topic.
At a fairly early stage it is useful to write a brief outline of your paper, organized according to some major headings (very often the ones taken up below will be useful) and with some notes under each heading.
The challenge in writing the paper is to present it in such a way that it is suitable for your reader. What do you need to explain? What can be taken for granted? It is often natural to skip the introduction and write the main body of the paper first. The first draft is ... a draft. Make sure that there is a logical progression in your paper, without gaps in the argument. Ask your teacher or a fellow student to read your draft. And be prepared to re-write!
There are at least two ways of presenting your investigation (and they can be combined):
- inductive: let the reader take part in the process of discovery; proceed gradually; do not start with revealing your results.
- deductive: you start with (a taste of) the conclusion, and show how you have reached it.
6 Some characteristics of linguistic discourse
To a linguist (as opposed to a literary scholar), describing the language of a geographical area, a period, a genre, or a writer, is something that may be worth doing as such. This is exactly what descriptive linguistics is about: describing the language as it is actually used, not as certain people think that it should be used. Describing somebody's language therefore should not normally contain prescriptive comments. (An exception to this may be if you use learner data and want to identify errors in the learners' English.)
By comparison, if you write about a novel or a poem, you will often have to make subjective interpretations which are not simply tied to words or combinations of words but to associations and experiences of your own which may go far beyond the text itself.
Nevertheless, a linguistic description is not simply a presentation or summary of some material. Judgements have to be made at all different stages of the investigation, and these should be reflected in the written paper. What sort of a problem are you dealing with? What material might be appropriate to use as a basis of the investigation? How should it be delimited? What method of analysis is appropriate? Example material may require interpretation. What patterns can you see? What do they mean? What is the effect of this or that construction? The results of the analysis must also be interpreted. How do they fit in with previous knowledge about the language? To what extent have you been able to answer your research question?
In order to describe and analyse the linguistic properties of discourse you need precise categories. You must also use the terminology correctly and consistently. Your secondary reading will provide guidance on terminology. Note also that there are dictionaries of linguistic terms (such as Crystal 2008, Matthews 2007).
There are situations where there is disagreement about terminology. Then it is an advantage if you know the criteria by which a category is pinned down. There are also occasions when you cannot rely on ready-made categories; sometimes you may have to define your own, in case the categories found in your reading are inadequate, or if the topic is very specific.
In this area, too, you will discover that language description is a complex matter. There are many instances of fuzzy categories. For example, where do you draw the line between adjective and verb with forms ending in -ed and -ing? How do you delimit phrasal verbs from related constructions? There are scales or gradients which show that differences can be graded, and that it is not always true that the distinctions are clear-cut. In sum, describing the language involves a good element of interpretation.
7 Supporting your statements
In your discussion,make reference to your own data and the secondary sources. When you quote, include as much as is necessary, but no more. If you need to abbreviate an example or a quotation from a secondary source, insert three dots (indicating ellipsis). Editorial comments can be added within square brackets. When you quote or paraphrase from somebody else's work, always give a reference to your source. In giving examples from your primary material, it is useful to number the examples (as above). This makes it easier to refer to them in your discussion. Each example should have an identification of the source. Look at the articles on your syllabus to see how this is done.
Distinguish between safe conclusions clearly validated by the data, and uncertain ones, for which you have inconclusive or incomplete evidence. Do not conceal data which may be difficult to account for. On the contrary, such material may require special comment.
8 Organizing the paper
Organize your paper into sections, with headings. This makes it easier to follow the steps in your investigation. It is often useful to number the sections (as in this paper).
8.1 Before the main text
On the title page, write the title of the paper, your name, the name of the department, the date (term), and the course your term paper relates to. Make sure the title of the paper reflects its aim and scope. If your paper is long, add a separate contents page. If you use a lot of abbreviations (apart from conventional ones), you should have a list of abbreviations. If your paper includes many tables and figures, include a list of these.
8.2 The main text
The main text can often be organized as follows (needless to say, the organization may vary depending upon the type of topic):
- Introduction. State your aim briefly and give the reasons why the subject of your paper is worth writing about. Give a brief, critical survey of earlier work dealing with your subject.
- Aim and scope. State your purpose in greater detail and tell the reader what aspect you intend to investigate as well as what will be left out.
- Material. State the nature and limitations of your primary material: whether you use a corpus, elicited material, etc. Describe your method of collecting data as well as the advantages and/or limitations of your material. Consider whether your choice of data is likely to affect the results in an important way.
- Method. (a) State along what lines your investigation will be conducted, and, if possible, give the most important sources of inspiration. (b) Define your terms and principles of classification. Explain your use of abbreviations and symbols (if these are numerous, give a list of abbreviations/symbols at the beginning of your paper). State any technical conventions you intend to follow. (c) If your investigation is long and complex, give a step-by-step description of what you intend to do.
- Analysis and discussion. First you present the results of your study of the primary material, and then you analyse them. You will probably want to present your results in the form of tables or lists of examples, or both. Focus on one aspect at a time. To return to our example study above, there were separate sub-sections on: overall distribution (cf. Table 1), length (cf. Table 2), definiteness (cf. Table 3), literal vs. figurative meaning (cf. Table 4), adverbial specification, and text category (cf. Table 5). Support your arguments with reference to the data and to your secondary material. Long and complicated sections should have a short summary at the end.
- Conclusion. To what extent have you been able to answer the question raised at the beginning of the paper? Give a general summary of your results and state the conclusions you can draw on the basis of them. To what extent do your conclusions agree with what was known before? If some of your results are inconclusive, e.g. because you have not had enough material, say so. Also indicate what aspects or areas require further study.
- Footnotes/endnotes. Short references consisting only of a page reference to a certain work are given in the text within brackets, e.g. (Quirk et al. 1985: 243f.). Slightly longer references and comments are given in footnotes (placed at the bottom of the relevant page) or endnotes (placed just before the list of references). Keep the length and number of footnotes/endnotes to a minimum.
- References. Under this heading you list your sources in alphabetical order according to the author's surname. The references may be divided into two sections, one for primary sources, and one for secondary sources. If you have used abbreviations for dictionaries or your primary sources, these should be clearly indicated in the list of references. Click here to see how you go about making references and reference lists. There is also a useful site here: http://sokogskriv.no/en/. An author-date referencing style is preferable in linguistics.
- Appendices. It is often useful to relegate some material to appendices, e.g. long lists of examples or detailed tables. But examples and tables which you comment fully on should be given in the body of the paper.
9 Some writing/formatting conventions
Writing about linguistic items: Use italic type to mark out a word or phrase that you are discussing and which is not an integrated part of the running text. Examples: (1) The most common word in the English language is the. (2) The word foot enters into a lot of more or less fixed phrases such as foot the bill, foot-and-mouth disease, put one’s foot in it and set foot in ….
Quoting examples: Use normal type for the example. However, you may want to add emphasis (for instance by using italics) for the part of the example that is most relevant to the discussion. Examples should be numbered for ease of reference. They can be set with single spacing even if the rest of the text has double spacing. See further advice on how to quote examples.
Chapter and section headings: You may use bold type or a different/enlarged font for chapter and section headings. Normally chapter headings are more prominent than section headings. In a term paper it will often not be necessary to subdivide sections (e.g. 2.1, 2.2, 2.3), but in an MA thesis this is often done, just as in linguistics books. Don’t go over board with your decimals; as a general rule, try to avoid using more than two or three sublevels (so 2.1.1 and possibly 184.108.40.206 are all right, but not 220.127.116.11.2).
10 Language use
Before you hand in your paper, make sure that it is free of errors in language (grammar, vocabulary, spelling). Check that pronoun references are clear: do not overuse this and that in referring to the preceding text. Make sure the verbs agree with their subjects. Avoid sentence fragments, without a subject and a verb. On the other hand, do not use long run-on sentences, with main clauses loosely strung together. If you can replace a comma by a full stop, do so. Be careful with paragraphing this contributes greatly to the clarity and coherence of your paper.
It is natural to adopt a fairly formal style. Avoid contractions (she's, aren't, etc.). Reduce reference to yourself to a minimum. Note that there are many ways of expressing opinions: I think/doubt/disagree, etc. may be regarded as too informal for an academic paper. Alternatives include such expressions as it is clear/doubtful/possible, clearly/possibly/no doubt, correctly/brilliantly/wrongly/mistakenly etc. Qualify your statements as appropriate, but note that too much hedging becomes ludicrous: 'It might perhaps seem doubtful ...'. Say what you think is true, but no more and no less. It may be a good idea to collect useful and/or elegant turns of phrase from books and articles about linguistics.
Finally, to avoid boring your reader you should try to vary your language (but not at the expense of clarity!). A synonym dictionary or a thesaurus may give you ideas for alternative ways of expression. The English language, the subject of your paper, is a rich and flexible instrument, and it is good if this shows in your paper.
- This paper is based on Stig Johansson’s manuscript "Writing a Term Paper in English Linguistics: Some Hints for Mellomfag and Hovedfag Students" (unpublished, , 1997). Sections 1, 3, and 7-8 of this paper are based on Altenberg et al. (1980). Section 6 draws on material provided by Kay Wikberg. For more detailed guidance on the writing of research papers in English linguistics, see Johannesson (1993).
Altenberg, Bengt, Jan Svartvik, and GunnelTottie. 1980. How to write a term paper in English linguistics. Department of English, . Unpublished.
Crystal, David. 2008. A Dictionary of Linguistics and Phonetics. 6th ed. Oxford: Wiley Blackwell.
Johannesson, Nils-Lennart. 1993. English Language Essays: Investigation Method and Writing Strategies. 4th ed. English Department, .
Johansson, Stig. 1986. Some observations on the order of adverbial particles and objects in the LOB Corpus. In Sven Jacobson (ed.), Papers from the Third Scandinavian Symposium on Syntactic Variation, , . Studies in English 56. : Almqvist & Wiksell. 51-62.
Matthews, P.H. 2007. The Concise Oxford Dictionary of Linguistics. Oxford: Oxford University Press. [Available at the UiOhere]
Scott, Mike. 2001. Comparing corpora and identifying key words, collocations, and frequency distributions through the WordSmith Tools suite of computer programs. In MohsenGhadessy, Alex Henry, and Robert L. Roseberry (eds), Small Corpus Studies: Theory and Practice, 47-67. : Benjamins.
Links to websites on term paper writing
top of page
© Stig Johansson/Hilde Hasselgård and the Department of Literature, Area Studies and European Languages,
Last edited October 2017, HH