Projects
See All My Projects
Admin

Sketch the Argument


Introduction

 There are a lot of things you need to think about when putting together a proposal.  In this technique, you'll use a "canvas" to sketch out the big picture:


The canvas helps you think through all the key pieces of the proposal so that you can:

  • see what you need to figure out
  • quickly find places where the pieces of your proposal don't line up
  • get feedback fast!

How

Materials

For this technique, all you need is a:

  • a google doc of the R&D canvas
  • (or if you can't access the google doc, you can create a slide using the canvas image or word doc instead)

Steps

Create a google doc with your canvas in it then follow the steps below to fill out your canvas.

1. Identify a learning situation

diagram1.jpg

To do learning sciences (LS) research and/or development (R&D), you need to choose a  specific and feasible problem to solve.  Unfortunately, it's really easy at the beginning to pick something that is too big or vague to solve.  Fortunately, it’s not too hard to start scoping an LS R&D, you just have to identify the learner and the problem solving task.


In the "learner" box of the canvas, write down:

  • Who is your learner?
  • What is the learner’s problem solving task
  • How would you measure if someone does the task well? (what knowledge, skills, dispositions should be shown)


Testing your learner and task

The challenge of scoping is to make sure that your learner and task are very specific.

Initially, make sure that the learner is specific enough that you could literally call up this person.  For example, “public school students” is probably too large a group for most projects.  “Jenny an 8th grade middle school student at Bessie Rhodes elementary” — is far more concrete and tells you much more about, age, demographics, location, cultural context, of your learners.  You can worry about generalizing later, but at the beginning it helps to be on the specific side.

Equally important is to be specific about the task.  “Understand math” isn’t specific enough.  Imaging you were going to hand teachers a test to give students and some guidelines about how to assess whether the student has performed well.  If you don’t think you could do that, you probably need to think more about the task you want learners to do.  Again, it’s easier if you start out being overly concrete because it’s much easier to generalize later than vice versa.


Double check:

  • You have identified your learner specifically enough you could call them up

  • Your learning task and assessment is specific enough you could write up a “test”


2. Is this a learning problem worth solving?  Really?

diagram2.jpg

Just because we want learners to perform a task well, doesn’t necessarily mean we need to teach them.  Some things people can figure out many things from everyday experience, while other things require designed learning environments.  For example, people learn to speak their first language without any curriculum, but learning a second language or learning to write usually requires education. That’s why you are more likely to see learning scientists focus on second language learning or writing than first language acquisition (although that might be really interesting to a developmental psychologist).

So to determine if you’ve really got  learning problem, you need to figure out what mistakes learners make, what cause the mistakes, and why they can’t learn from everyday experience.


In the "learner" and "expertise/root causes" boxes of the canvas, write down:

  • The mistakes learners make

  • What causes the learner to make mistakes

  • Why they can’t learn from everyday experience


Testing if this is really a learning problem

To test if you’ve got a real learning problem, ask yourself:

(a) Do learners make mistakes?

If yes, keep going, if no, choose a different learner/learning task

(b) What are the causes of these mistakes?

If the task is: complicated, counter-intuitive, fuzzy, very abstract, tedious, or stressful, keep going!

If not, i.e., you have a simple task and we really just need to tell learners  something basic information or giving them instructions or pointing them to a useful resource, then it’s not really an serious learning problem that you need to devote a lot time to solving (i.e., you’ve already solved it!)  In that case, focus on a higher-level task.  

If you think the cause of the mistake is “we aren’t teaching them” then you need to think harder about the real causes—people learns all kinds of stuff without being taught explicitly, what makes this task hard?

(c) Why can’t people learn from failure?

If the task is such that:

  • a mistake would cause death or harm

  • everyday experience doesn’t provide useful or usable feedback E.g., if you write a bad resume, you don't find out why, you just don't get any interviews.

  • the situations that lead to mistakes, while important, don't occur very often.

then you probably have a learning problem that requires education.

If the task is such that people will learn just fine by muddling through everyday experience, then it’s probably not worth the hassle of creating a learning environment, so pick a more complicated task.


Double check that:

  • The causes of the mistakes are that the task is cognitively complex

  • The mistakes can’t be remedied by just giving people instructions or “just teach it”

  • People can’t learn from failure because everyday experience is dangerous, doesn’t provide useful feedback or mistakes are important but infrequent


3. Who cares (practically) about this learning problem?

diagram.jpg

You want to do Learning Sciences R&D that matters or what NSF calls broader impact.  Part of figuring this out involves determining what practical audience  actually care about what you are doing.  Unfortunately, this can get a little tricky because your R&D involves keeping track of bunch of different audiences that will trip you up if you aren’t careful.

In LS research & development, there are a number of audiences/perspectives we have to think about:

  • The learner
  • Stakeholders who care about the learners’ performance
  • Instructors who support the learners
  • Researchers / thought leaders who build knowledge

One way this can be confusing is that different people can occupy different roles or even multiple roles.  Let’s look at some common examples:

Example 1: The Classroom. Let’s say that we want to do R&D on teaching 8th grade geometry.  Then our audiences are:

  • learner - 8th grade geometry students
  • stakeholders - Illinois citizens, to the extent that our society has agreed that it’s in our collective best interest that 8th graders can do geometry, as evidenced by state mathematics standards.
  • instructors - the middle school math teachers that teach geometry
  • researchers - mathematics education researchers that build theory about learning and teaching mathematics; district managers in charge of curriculum and instruction

Example 2: Teacher training.  Let’s say that we want to do R&D on a high school social studies teacher training, where the teachers are the learners.  In that case, the teacher will become the learner so our audiences are:

  • learner - high school teacher trainers
  • stakeholders - Illinois citizens, to the extent that society has agreed we need teachers who can help learners reach common core standards
  • instructors - the social studies teacher trainers, such as social studies coaches or expert teachers that conduct social studies in-service trainings to teach teachers
  • researchers - social studies education researchers interested in teacher education; district managers in charge of curriculum and instruction.

Example 3: Informal learning.  Let’s say that you want to do R&D about how people learn “in the wild” focusing on young adult makers who are learning  cosplay.   In that case, the young adult plays several roles and our audiences are:

  • learner - young adult cosplayer
  • stakeholders - young adult cosplayer — there isn’t really anyone who has a vested interested in the young adult’s cosplay ability except the cosplayer!  Even cosplay convention organizers don’t really care if the cosplayer displayers expert-level ability.
  • instructors - other cosplayers — cosplayers mostly learn from others in the cosplay community (I think).
  • researchers - new media and connected learning educational researchers interested in how people learn outside of school.


In the "stakeholders" box of the canvas, write down:

  • the (most important) stakeholders interested in learners’ performance

  • why the stakeholders care

4. Why do your stakeholders care (really)?

One of the biggest problems in research & development is doing work that no one cares about.  Just because you've identified a stakeholder doesn't mean the stakeholder sees value in your proposal.  

You can be more explicit by writing a "value proposition" for you work to the stakeholder (which you can later test) using this value proposition mad lib:

Our <product/service/work> help(s) <stakeholder> who wants to <job to be done> by <avoiding some pain> and/or <achieving some gain  (unlike <competing value proposition>).


In the "value proposition" box of the canvas, write down:

  • Stakeholders or Instructors (i.e., the people involved in carrying out the solution) and why they want this learning environment or research



Testing for broader impact

To test for broader impact, you just need to check that some stakeholders cares about the mistakes learners make on this task.  The best way to test your value proposition is to show it to stakeholders, (such as instructors or funders) and see if they will provide some resources (their time, labor, money) in exchange for something that delivers that value.  If that's not possible, show it to someone who has some experience with the stakeholder and ask them to predict whether they think the stakeholder will react.


Double check that:

  • stakeholders really care about learners’ performance (and we think these are important stakeholders!)



5. How might we solve the problem?

diagram.jpg


Now that you’ve got a practical problem, how are you going to solve it?

Think about how your solution will address:

  • Setting goals for the learner

  • Motivating learners

  • Providing necessary supportive information e.g., conceptual schema and problem-solving approaches

  • Practice opportunities

  • Feedback

  • Cultural norms


In the "value proposition" box of the canvas, write down:

  • Instructors (i.e., the people involved in carrying out the solution) and why they want this learning environment

In the "design argument" box of the canvas, write down:
  • Your design argument, i.e.

    • your solution, focusing on the core ideas by which it works

    • the instructional principles upon which your solution is based


6. Who cares (intellectually) about this learning sciences problem?

diagram4.jpg

If you are an academic, innovator or thought leader in LS, you also want to do work that has intellectual merit.  

Specific intellectual audience

The first problem you will run into is being specific about your intellectual audience.  This is an easy fix though — just specify which journal, conference, blog, etc. that you are going to write for.

If you are writing for academic learning sciences audience and don't know which journal is right for you, consider starting with one of the 8 sister journals in the learning sciences described on the ICLS journals page.


In the "venue" box of the canvas, write down

  • The venue you will publish in to reach your intellectual audience


Testing your audience

Simple — just name the venue!


7. The research question

The next difficulty is precisely stating your research question.  There are many possibilities but here are some very common forms:

  • What knowledge, skills, dispositions are required to do X?

  • What are the challenges involved in learning X?

  • What factors influence learning X?

  • How might we design a solution X to promote learning Y?  or Does solution X promote learning Y?

  • How can we model learning phenomenon X (that are difficult to empirically observe?

  • How can we build tools that help us conduct research (that is difficult or time consuming to do).


Write down:

  • Your research or development question


Testing your research question

Is it novel?

Unlike the practical audience, the intellectual audience is only interested in questions that NO ONE knows the answer to.  If someone has already answered the question, it has little intellectual merit.  This isn’t as scary as it might seem—there are many variations of domains, learners, solutions and contexts, so there is always much more that we don’t know .

Is it an LS problem?

Part of this requires being clear about what kinds of problems LS can actually investigates.  There are many important educational problems we want to solve, but not all of them are problems that can be solved by learning sciences research.  Often you must know we must know and write about (as citizens) that require other kinds of theory and research methods.  For example:

(a) productivity problems - how do I make tools that increase novice performance — this a great question, but it’s not really a question about learning.  If you want to turn it into a learning question, you might want to look at how people learn to use the new tool.

(b) policy, politics & ethics problems- What should schools teach?  How do we change school standards?  These are crucial educational questions that you must make arguments about, but LS research methods don’t help you construct these arguments.  These are questions solved by ethics and decided via political processes (which is why you should drop whatever you are researching now and switch to civic education!).  To turn a policy question into a learning sciences question, you can either ask how people learn/teach the domain you are interested in, or perhaps treat the decision-maker as the learner!

(c) personal questions -  I don’t know how students learn second languages;  I don’t know the best way to create an online learning community.  These are good starting points for R&D projects, but your next step is to figure out what the research community has figured out about these questions.  In many cases, you’ll find that other people have asked the same thing and have some good answers!  To turn personal questions into LS research questions, you have to propose a novel question that someone hasn’t figured out before.

Is it an empirical problem?

LS research questions are empirical questions. LS theory and research methods can only really address empirical questions about learning and teach.  For example, How might we better teach 9th grade geometry?  or How effective is secondary social studies teacher training? or How do young adults learn cosplay in informal online communities? or What abilities are required to become an expert-level graffiti artist?  These are all empirical questions that LS research theory and methods can help you solve.   If your question can eventually be resolved through empirical data you probably need to reformulate.


Double check that:

  • Your research question is novel

  • Is a problem of learning

  • Is an empirical problem