Research Methodologies

Descriptive research
Purpose:

1. To describe a phenomenon, for example:
To what extent do the tenth graders use a dictionary in reading comprehension?
How can computer games be used to teach vocabulary to the fourth graders?

2. To explore a phenomenon, for example:
What were the obstacles in implementing international standard in SMA 3?
Why did SMA 3 face those obstacles?
How did SMA 3 overcome the obstacles?


Experimental research
Purpose: to determine cause & effect, for example:
Does the use of L1 subtitles have an effect on the tenth graders’ listening comprehension?
Is there a significant difference in speaking ability between fifth graders who are taught grammar and those who are not?


Ex Post Facto research
Purpose: to predict cause & effect, for example:
Is there a significant difference in speaking ability between university students who learned English in primary schools and those who did not?
Do the students who are extrovert learn English better than those who are introvert?

Sampling

Population is all people we want to investigate.
Sample is a part of the population.
Q: Why should we draw a sample from the population?
A: Because the population is too large and should be reduced to enable investigation.


Types of Sampling

1. Probability sampling
We draw the sample randomly. This type of sampling is used in quantitative research, but never used in qualitative research.

a. Simple random sampling
It means everybody has an equal chance of being selected as a sample. You draw the sample by choosing individual students randomly. For example, you have 100 students in the population so write the students' names on 100 small pieces of paper. Fold the pieces of paper and put them in a bowl. Shake the bowl, then take 30 pieces of paper randomly from the bowl. Now you have 30 students as the sample of your study.

b. Stratified sampling
The population is divided into several strata/levels, then the sample is drawn from each level. For instance, you want to conduct research at SMAN 5. There are three grades: the 10th grade, the 11th grade and the 12 grade. These grades are the levels, which can be used as a basis to draw the sample. List the number of the students from the highest to the lowest level, then draw a sample from each level.

c. Cluster sampling
The population is divided into several clusters/groups, then a sample is drawn from each group. For example, you want to conduct research to the tenth graders. If there are five classes (clusters), list the number of the students in these classes and draw a sample from each class.

d. Systematic sampling
The sample is drawn by selecting the n-th person in the population. For example, there are 150 students in the population and you need 30 students in your sample. Divide the number of students in the population (150) by the number of students in the sample (30), you get 5.

150 : 30 = 5

It means you could draw the sample by choosing every 5th student in the list.

2. Non-probability sampling
We do not draw the sample randomly. It is used in qualitative and quantitative research.

a. Accidental sampling
Choosing any person that you meet as the sample.

b. Purposive sampling
Selecting a sample from the population for a particular reason. This is used in qualitative research, but not quantitative research. For example, you conduct research to find out the strategies of proficient readers in comprehending a narrative text. You need to draw a sample from a reading class, and choose 8 proficient readers by taking the teacher's judgement into account. Ask the reading teacher which students are proficient readers, then you could select 8 out of several students mentioned by the teacher.

c. Quota sampling
Dividing the population into levels, then selecting n % from each level. For example, you list the 11th graders belonging to programs A1, A2, A3 and A4. Because the number of students in these programs varies, it is better to use quota sampling by taking 10% of students in each program as your sample.

Some terms

There are some terms you need to know when you conduct research and write a thesis. Here they are.

ooOoo

A concept is a word that refers to an object or an event.
Example: book, table, jump, slow, loud, shiver.
You can perceive them using your senses. You can see a book or a table, you can see someone jump, you can see a slow thing, you can hear something loud, etc.

A construct is a psychological concept.
Example: motivation, intelligence, reading ability, language acquisition.
You cannot perceive them using your senses because they are 'psychological'. Even if you open someone's head, you cannot see their intelligence, motivation, or reading ability. But they exist!

ooOoo

Constitutive definition: a word is defined by using other words.
Example:
A student is a person studying in a school or a university.
Conference proceedings can be defined as a collection of scientific papers presented in a conference.

Operational definition: a word is defined by mentioning the operation that must be done to measure the concept/construct.
Example:
Intelligence quotient is defined as the scores on the Stanford-Binet Test of Intelligence.
English proficiency is defined as the scores of Test of English as a Foreign Language (TOEFL).

ooOoo

A variable is something that varies.
Example:
Age can be 12 years old, 38 years old, 50 years old, etc. Age varies so it is a variable.
Gender can male or female. Gender varies so it is a variable.
English proficiency can be good, average or poor. English proficiency varies so it is a variable.

Quantitative and Qualitative Research

Quantitative and qualitative are two differrent approaches to research. In fact, they are very, very different, like fire and ice. Theoretically, you cannot do both in a single study. However, in practice sometimes it is possible to use both approaches in a study (see mixed method in a previous entry).

I have explained how the two differ in my 'Research Methodology' lectures, and now I post the table showing their differences below as you said you needed my slides. (If you think the size of the letters is too small and you find it hard to read, just click the image and you will see the larger version. You can save it to your hard drive.)


In general that's how they differ. The procedure of conducting both research is actually also different. Please see the image below.


When you conduct quantitative research, you begin by stating the research questions, and based on them you formulate hypotheses. After that, you collect data, analyze them, and report the findings which result from the analysis. So the process is quite linear, like a straight line.

When you conduct qualitative research, you begin by collecting preliminary data in the field. You have to go to the field to collect data, and based on the data you could state the research questions. Next, you return to the field and collect more data. It is possible (not always) that you do not get the data you want. In this case, you have to revise the research questions. That's why I draw an arrow connecting research questions and data collection in the above picture. After revising the research questions, you can go to the field again and collect data. Then you could analyze the data and report the findings. The process of conducting qualitative research can be (again, not always) like a spiral.


If you have questions, just write them in the comment box. I'll try to answer them as soon as possible.

Experimental & Ex-post Facto Research

In the last session I provided a brief overview of the experimental and ex-post facto designs. Basically here is how they differ.

Experimental

Research question: Are songs effective to teach listening comprehension?

Design:

You draw the sample randomly from the population.
Then you assign the sample into an experimental and control groups also randomly.
You give a treatment to the experimental group, but not to the control one.


Ex-post Facto

Research question: Does learning English in elementary school affect the university students’ speaking ability?

Design:

There is no randomization here.
You select the subjects on the basis of a certain existing condition.
You do not give treatment to any group because:
1. the “treatment” already happened in the past and you can no longer do anything about it, or
2. it is unethical to give a treatment to the ex-post facto group.

We will discuss more about the above designs near the end of this semester.

Some issues about research

In the last session you asked me lots of good questions about research. I need to post the issues you addressed here.

1. Terms
These terms are synonymous: research, study, investigation, inquiry.
So in this context, study is not the same as learn. Rather, it refers to research.

2. Quantitative vs. Qualitative
Which one is better? Good question! The answer is: both.
The best research is the one that can provide the most appropriate answer to your research questions (problems). If you are wondering whether your teaching technique is effective and want to find out how many students can succeed when taught using that technique, then you must conduct quantitative research. If you conduct qualitative research to find out about this, then you're heading for the wrong direction. In this case, quantitative research is better than qualitative one.
But if you want to know how to apply a certain teaching technique in the classroom, then you must conduct qualitative research. In this case qualitative research is better than quantitative one.

3. Quantitative and Qualitative
One of you asked whether you could use both the quantitative and qualitative approaches in a single study. Good news, the answer is: yes. It's called mixed method. You need it when you have, for example, the following research questions:
1. How can songs be used to teach English to the 5th graders at Mbebekan elementary school? (qual.)
2. Can the use of songs improve the 5th graders' vocabulary size? (quan.)

4. Observation
The example of quantitative & qualitative observation can be downloaded here.

5. Questionnaire
The example of quantitative & qualitative questionnaire can be downloaded here.