Action Research

I have posted a few things about Action Research in my Student Teaching blog, so I won't repost it here. Click here to see the blog entry about Action Research.

Experimental Research

You have learned about the difference between experimental and ex-post facto research here and here. Let me review them a little bit.

Experimental research is conducted to determine cause and effect. For example, the topic of your research is:

The effects of using songs in the teaching of vocabulary on the students' vocabulary size.

You want to find out whether the use of songs (the cause) has effects on the students' vocabulary size. In this case, the independent variable is the use of songs, while the dependent variable is the students' vocabulary size.

The design of the experimental research is like this:


You have two groups, experimental and control groups. You give both of them a pretest, which is a vocabulary test, at the beginning of the semester. Make sure that both groups are equal. After that, you teach vocabulary to both groups for one semester. You teach the experimental group using songs (treatment), while the control groups is taught vocabulary without using songs. At the end of the semester you give both groups a posttest, which is exactly the same as the pretest, and analyze the means of the posttest using t-test to find out whether there is a significant difference between teaching vocabulary using songs and without using songs.

Characteristics
There are 3 characteristics of experimental research.

1. Control
The control is applied to the condition of the subjects.
You have an experimental group, and you have a control group to make sure the treatment makes (or doesn’t make) a difference to the experimental group. You control the condition of both groups. Both should:

• be of the same age
• have equal ability
• consist of equal number of males & females
• etc.

2. Manipulation
This is not a negative term in experimental research!
It simply means the manipulation of the independent variable.
The independent variable of the research problem "The effects of using songs on the students’ vocabulary size" is the use of songs. You manipulate this variable by doing this:

• The experimental group is taught using songs.
• The control group is taught without using songs.

Songs and no songs, that's the manipulation.

3. Observation
This is not a qualitative observation!
It simply means you observe the difference between the experimental and control groups after you conduct the experiment. You have to do statistical analysis (t-test) to find out whether there is a significant difference between the two groups.

Selecting the sample
You need two processes to select the sample of your experimental research.

1. Random sampling: selecting the sample from the population randomly, e.g.

10 classes of eight graders (population)

2 classes of eight graders (sample)

2. Random assignment: assigning the sample into 2 groups (experimental & control) randomly, e.g.
Class C experimental group
Class F control group

The above processes can be described in the chart below.

Research Methodologies

Descriptive research
Purpose:

1. To describe a phenomenon, for example:
To what extent do the tenth graders use a dictionary in reading comprehension?
How can computer games be used to teach vocabulary to the fourth graders?

2. To explore a phenomenon, for example:
What were the obstacles in implementing international standard in SMA 3?
Why did SMA 3 face those obstacles?
How did SMA 3 overcome the obstacles?


Experimental research
Purpose: to determine cause & effect, for example:
Does the use of L1 subtitles have an effect on the tenth graders’ listening comprehension?
Is there a significant difference in speaking ability between fifth graders who are taught grammar and those who are not?


Ex Post Facto research
Purpose: to predict cause & effect, for example:
Is there a significant difference in speaking ability between university students who learned English in primary schools and those who did not?
Do the students who are extrovert learn English better than those who are introvert?

Validity and Reliability

Reliability is the consistency of a test in measuring something.

If you give the same test to the same student three times and the results are 80 - 65 - 90, the test is not consistent in measuring the students' ability. The test has low reliability.
If you give the same test to the same student three times and the results are 85 - 85 - 85, the test is consistent in measuring the students' ability. The test has perfect reliability.

Reliability is not black and white, you cannot say the test is either reliable or not reliable. Instead you have to express the reliability in the form of degrees: very low, low, moderate, high, very high, perfect. (Remember about this when we talked about correlation in the statistics classes?)

Methods of estimating reliability are:

1. Test-retest: one test, administered twice.
2. Equivalent form: two similar tests, administered once.
3. Split-half: one test, administered once, split into two parts (odd & even numbers)
4. Internal consistency: one test, administered once, apply KR 21 formula.
5. Interrater reliability: one test, administered once, scored by 2 people.

Which method should you use?
If you use an objective test in your skripsi, I suggest internal consistency (KR21).
If you use an essay test, I suggest internal consistency (Cronbach Alpha).
If you use a writing test, I suggest interrater reliability (Pearson r).

Validity refers to the relevance of the content of test in measuring something.

If you use a math test to measure the students' English proficiency, the test is not valid.
If you use a reading test to measure the students' writing ability, the test has very low validity.
If you use a reading test to measure the students reading ability, the test has very high validity.

Methods of estimating validity:

1. Content validity: the test should representatively contain the items that are supposed to be measured. Map the content of your test against the content of the curriculum/syllabus/lesson plans. If they match, your test has high content validity.
2. Criterion-related validity: the content of a test is related with another test (a criterion). Correlate the results of your test and the results of a standardized test (the criterion). If the correlation is high, your test has high validity, too.
3. Construct validity: the test should measure the construct well, e.g. a reading test should measure the construct 'reading comprehension', a grammar test should measure the construct 'grammar ability', etc.

Criterion-related validity can be measures in two ways:

1. Concurrent validity: Administer your test and the standardized test at the same time, e.g. administer your test and the standardized test today.
2. Predictive validity: Administer your test, then administer the standardized test after several months have passed, e.g. administer your test today, administer the standardized test next semester.

Construct validity can also be measured in two ways:

1. Pilot study: administer the test to two groups of subjects, one group has the construct while the other does not have the construct. If the former scores higher than the latter, the test has construct validity.
2. Intervention study: administer the test to a group who does not have the construct as a pretest. Give treatment to the group by teaching the construct, then give them a posttest. If there's a significant difference between the results of pretest & posttest, the test has construct validity.

Which method should you use?
If you use an objective test in your skripsi, I suggest content or concurrent validity.
If you use an essay or writing test, I suggest content validity.

Research Instruments

Research instruments are the tools that you use to get data (words or numbers).

The instruments commonly used in educational research are: tests, questionnaires, interviews and observations.

Tests are categorized into two types:

1. Objective tests, such as true-false, multiple choice, matching, completion, etc.
2. Subjective tests, such as essay and writing tests.

If you use a test as the research instrument, you need to estimate the reliability and validity of the test. These characteristics of the test will be elaborated in a different entry.

Questionnaires are categorized into two types:

1. Structured: It has a fixed number of questions and a limited choice of answers. The respondents should choose one or more suitable answers. The aim of administering this questionnaire is to get facts.
2. Unstructured: It consists a number of questions. The answers are in the form of essay and the content may vary depending on the respondents. The aim of administering this questionnaire is to explore something.

Interviews are categorized into three types:

1. Structured: It has a fixed number of questions and limited choice of answers. It's like a structured questionnaire, but the questions are spoken by the researcher to the respondents.
   
2. Semi-structured: It has a few questions as guidelines, and can be added when necessary.
3. Unstructured: There are no questions at all in the beginning. The researcher determines the questions after immersing in the field for some time. The things to be asked will appear after the immersion.

Observations are categorized into two types:

1. Participatory: The researcher participates in the activities of the subjects, e.g. You conduct a study in SMA 1 and replace the teacher in the classroom. You teach and observe the students.
2. Non-participatory: The researcher does not participate in the activities of the subjects, e.g. You conduct a study in SMA 1. You only sit in the back of the classroom and observe the teaching-learning process.

When you observe your research subjects, you need one of these two things to help you collect data:

• Checklist: a list of things that you look at when observing something/someone.
• Field notes: anything you write when observing something/someone.