Projects
See All My Projects
Admin

Develop a study


Introduction

In a innovation/research proposal you propose a goal-knowledge conflict where you define the problem as some piece of knowledge we need to develop and your solution is some proposal on how to figure out that knowledge, such as by studying a situation or testing a design. To do this, you propose a study.

What are the elements of a study?

Background

Examples

Mosteller (2004) as well as APA (2010) describe the high level elements you need to develop for a study:

  • Setting: 76 public elementary schools drawn from inner-city, urban, suburban, and rural locations in Tennessee. A total of 328 kindergarten classes and 347 first-grade classes participated in the study.
  • Participants: 6,570 students enrolled in kindergarten in the 1985–1986 school year.
  • Intervention: Students were randomly assigned by project staff to one of three class types: small (13–17 pupils), regular (22–25 pupils), or regular with a teacher aide (22–25 pupils). Students assigned to small classes stayed in small classes for kindergarten and first grade.
  • Research Design: Randomized-controlled field trial.
  • Data Collection: The Stanford Achievement Tests in reading and mathematics were administered in the spring of each school year, and a set of Tennessee curriculum-referenced tests were administered at the beginning of first grade. 
  • Analysis: Means on each outcome measure were calculated for each class, then separately for White and minority students in each classroom. Two analyses were conducted using multivariate analysis of variance: a cross-sectional analysis of the entire first-grade sample and a longitudinal analysis of a subset of pupils (n = 2291) who were in the study for both kindergarten and first grade and had complete SAT achievement test data. 

Knowledge

At a high-level, the elements of a study design are pretty straightforward.

Setting
Where will the study take place?  While you will typically anonymize for the readers, you should be able to name a specific schools, location, lab etc.  That is, you should plan the setting in enough detail that you show exactly where it is on a map.

Participants
Who will participate in the study?  Similar to the setting, while you will anonmize it for the readers, you should know the participants in enough detail that you could call them up on the phone, identify them in a video, etc.  You should know the characteristics of the participants that are important for generating the evidence you need, such as demographic data, ability level, etc.

Intervention

The intervention describes, in concrete terms how you implemented in the design argument.  Whereas the design argument is phrased in terms of general principles, the intervention is a specific instantiation of the design argument.

(study) Design

The (study) design describes the way you organize participants into groups for study.  For example you might conduct a case study, an observational study, survey, an experiment, or a design-based research study.  

  • In a case study, you observe 1 naturally occurring group, such as studying how a particular teacher teaches their class.  
  • In an observational study you might observe multiple participants doing a similar task or test. 
  • An interview or survey is a kind of observational study except you ask each participant questions.  
  • In an experiment, you randomly assign people to different groups and give each group a different intervention.  
  • In a design-based research study, you do multiple iterations using one or more of these other study designs.

Data collection 

Explain how you will collect data.  When planning your data collection, you should know exactly when, where, how you will generate each data source.  This includes both qualitative and quantitative data.  For example, you might:

  • Interview people
  • Survey
  • Observe people using field notes or video
  • Measure performance on some task
  • Collect log data from the web or other sensors
  • Simulate -- if your study involves developing a model, you might generate data from running a simulation

Analysis

Analysis describes what you will do with the data once you have it in order to generate answers to your research question.  At a high level, analysis is just about what you are going to do to examine the relationships between variables.  In exploratory work, you may be trying to discover the variables and relationships that explain some phenomenon, e.g., "what factors might cause X?"  In deductive work, as in hypothesis testing, you have a guess about the relationship between variables and test whether that relationship exists, e.g., "I think X causes Y."   So in your analysis, you want to describe what kinds of relationships you are going to be examining.

You then want to go a little bit further and say what analysis techniques you will use to examine these relationships.  Some common types of analyses include:

  • Open coding -- tagging qualitative data with "emergent" tags to identify commonly occurring sets of tags "themes"
  • Closed coding -- tagging qualitative data with a pre-specified set of tags to see how well the tags account for the data.  This is often done after open coding to test the comprehensiveness of the coding scheme.  Both open and closed coding turn qualitative data to quantitative data.
  • Exploratory data analysis and descriptive statistics -- creating graphs and charts of quantitative data to answer questions about how much or which is more.
  • Statistical tests -- using statistical methods to determine whether differences in quantitative data are significantly different.
  • Machine learning -- fancy statistical techniques that rely on computation to discover patterns in the data that aren't detectable to the human-eye.

References

American Psychological Association. (2010). Publication manual of the american psychological association (6 ed.). Washington DC: American Psychological Association.

Mosteller, F., Nave, B., & Miech, E. J. (2004). Why we need a structured abstract in education research. Educational Researcher, 33(1), 29-34.

Trochim, W. M. K. (2005). Research methods: The concise knowledge base. Cincinnati, Ohio: Atomic Dog Pub.