Projects

See All My Projects

Admin

Crowd Feedback

HCW build effective systems for Crowd Feedback? Does it work better than other feedback?

Crowd Feedback in Delta Lab

Team

You must be logged in to add comments.

Stand for Wednesday, Feb 24

What’s your sprint goal for this week?
- fix IRR for individual skills
- complete grading, IRR, and t-test of group blueprints
- write up group blueprints section for demo

We also wanted to get a head start on our next demo, which is to begin to grade individual skills tests from Fall 2015.
Are you on track to achieving your sprint goal?
Yes
What specific progress have you made toward the goal since last stand?
We've finished everything but the write up. We're behind on starting to grade individual skills test, which will push us back for our next sprint possibly, but not for Friday.
What will you achieve between now and the next stand?
Complete the write up for the demo.
What obstacles are in the way of achieving your sprint goal? Who do you need to talk to or what will you do to overcome these obstacles?
Things look good for now.

Emily Harburg - Keep it up, Gulu & Phoebe!

Hope you rest up and feel better, Gulu!

2 years ago

Gulnaz Saiyed

Review from last spring:

Pluses: Teamwork continues to go well! It's great that the project maintains momentum even though I wasn't able to be here on Friday. Our IRR is finally where it should be for the individual tests, and I think that we're finally getting a good sense of how to communicate with each other about evaluating consistently.
Minuses: We didn't evaluate consistently on the blueprints, it seems like we need to be more solid with our rubric (though getting feedback on this was good!) so that there is less grey area about what does and doesn't work. I think part of the challenge here is that we weren't on this project from the inception - that said, I think if we get it to the point where it makes sense to both of us in the same way, then it's more successful than if it just made sense to the designers. Also, I didn't get to the rewrite that I wanted to do if the individual results yet, so had to move it to the next sprint.

Our new goals are at the workbench.

2 years ago ·

Gulnaz Saiyed - *sprint

2 years ago

Stand for Friday, Feb 5

What’s your sprint goal for this week?
As a reviewer, I can see the mean score on teams' project blueprints before the intervention, so I that I can decide how much students know before critique. [5]

As a reviewer, I can see the mean score on teams' project blueprints after the intervention, so I that I can decide how much students know after critique. [5]

As a reviewer, I can see the interrater reliablity so that I can decide if the coding scheme was objective. [3] [double-check the Cohen article, may involve re-rating]
Are you on track to achieving your sprint goal?
No
What specific progress have you made toward the goal since last stand?
We've started grading the pre and post Blueprints and made a rubric that we're happy with.

What will you achieve between now and the next stand?
finish grading and a t-test
What obstacles are in the way of achieving your sprint goal? Who do you need to talk to or what will you do to overcome these obstacles?
We are behind the revamping the inter-rater reliability score because need to get to SPSS, so this might put us behind a little. Phoebe is going to talk to IT at SESP early next week for this.

Daniel Rees Lewis - T-test! Good job :)

2 years ago

Gulnaz Saiyed

Value Props:
There are my working value propositions. The first one draws most heavily from the outlined paper/proposal that Matt (I think) wrote. The second one is essentially the same thing, but in my own words. I'm not totally sure of the level at which I should be writing these. I felt like I could both zoom in or out.

Our feedback paper helps researchers and practioners who want to increase learning in problem-based learning environments by describing the design of a research-based hybrid crowd critique system and demonstrating its efficacy in supporting students’ learning of feedback-giving skills.

Our feedback paper helps researchers and practitioners who want to design environments in which students learn to solve complex, authentic problems by demonstrating the need for and effectiveness of a hybrid crowd critique system that addresses the challenge of providing learners with immediate, high quality feedback.

2 years ago ·

Daniel Rees Lewis - Nice! I think I have the same slight difference with Matt -- that the value is more about the learning to do the thing you're giving feedback on, not learning to give feedback (but I might be out of touch with this paper).

I did read an interesting paper from France in which there were higher learning gains in a group which _gave_ rather than received feedback (as I remember it was middle school students learning to write essays). So perhaps there might not be that much difference in the two, as although they can be de-coupled learning to give feedback might in this context also mean learning to do the thing.

2 years ago

Gulnaz Saiyed

User Story Stem: As a user XXX , I would like to XXX so that XXX

2 years ago ·

Gulnaz Saiyed

roadblocks/quesitons for matt on grading blueprints:
- from Crit v4 Blueprint, what is "Analysis of the student ratings of work quality (based on hashtags)"
- are we (Phoebe and I) the expert raters? or is it your (Matt's) feedback on loft?
- is "Measure 3 Critique checklist" our rubric? (not sure I ever received a final word on this)
- we're not looking at master design docs, right?
- can we just walk through the "Crit v4 Blueprint" so that we are all on the same page about what happened when and compare it to the data we have in Box?
- "Crit v4 Blueprint" says to focus on one group -- will we narrow it down, or keep with all of them?

On individual skills tests:
- need help with IRR
- what do we do with it now? (write it up and put it to the side?)

2 years ago ·

Stand for Monday, Jan 18

What’s your sprint goal for this week?
By this Friday, we will have completed inter-rater reliability of our data and run statistical tests on individual change for each item on the blueprint.
By next Friday, I will have graded all the pre and post Blueprints and present a list of initial insights for the demo.
Are you on track to achieving your sprint goal?
Yes
What specific progress have you made toward the goal since last stand?
NA
What will you achieve between now and the next stand?
- Re-done the pre/post individual grading
- Done 1 reading on IRR to get an idea
- Review rubric for group blueprint
What obstacles are in the way of achieving your sprint goal? Who do you need to talk to or what will you do to overcome these obstacles?
- do IRR with Phoebe and Matt
- make sure we're grading the group blueprints correctly

Natalia Smirnov - Go Gulu & Phoebe!!

2 years ago

Phoebe Pahng joined the project

2 years ago ·

Gulnaz Saiyed joined the project

2 years ago ·

Natalia Smirnov

Figuring out how to do SNA analysis... Created Matrixes for Start Feedback Network Condition. Mostly positive relationships except for a few that likely indicate conflcited relations with a person/mistrust in their intelligence. This has interesting implications. Need to look in undergrad class to see the same thing.

Next steps:
- Read example papers
- Learn how to do SNA analysis in R
- Make matrices and generate measures for 1st and 2nd rounds!


R SNA RESOURCES:
- http://web.stanford.edu/~messing/RforSNA.html
- http://sna.stanford.edu/rlabs.php

SNA general:
- http://faculty.ucr.edu/~hanneman/nettext/C18_Statistics.html

3 years ago ·

Natalia Smirnov joined the project

3 years ago ·

Natalia Smirnov

3 years ago ·

Natalia Smirnov

Looking over test "Impact Map" tests:

- Many critiques on part 3 are very vague, like "doesn't seem right" - "this is confusing" - not USEFUL
- or almost useful but not specific - "lines could be nicer" -- HOW?
- people don't tend to list that many critiques
- part 2 - people seem to list the major components, though most don't have metrics; many have GOALS instead of behaviors.. which is kinda different...
- one is interesting - only has goals/actors in part 2 - but has really extensive feedback in part 3
- part 1 - answers seem pretty general - like "understand" or "give feebback"


3 years ago ·

Edit Story
File Upload
Rename File
Send Invitations

Type in names or emails, or paste in a list separated by commas

Edit Event
Goal Instruction

Loading ...

Edit Due Date
Take me to the Loft

Welcome to the Loft!

The Loft is a suite of tools created to help you run successful design projects.

Take me to the Loft

Projects

The Loft has built-in tools to make sure your design project runs smoothly.

Click your project name in the upper right corner to quickly navigate to your project.

The Loft has built-in tools to make sure your design project runs smoothly.

Take me to the Loft

Goals

Loft goals help keep your team on track and always knowing what the next step will be.

See your next deadline and statuses on goals.

Choose which goals you want to take on from the goal libarary.

Take me to the Loft

Feedback

With the Loft feedback system you can get real-time comments on your work from your peers.

Upload your project work for your peers and mentors to review.

Get real-time feedback on your work.

Take me to the Loft

Guides

Loft guides give you a framework of activities to help your team through a design process.

Browse the guides to choose the best process for you.

Take me to the Loft

Studios

The Loft has a community of studios from around the country, each with their own project teams and guides.

Take me to the Loft

Get Started

You now have access to a powerful suite of tools to help you design successfully. Have fun!

Don't worry, if you ever get stuck you can always click the help button to get answers.