Summary of Evaluation Critique Team Results

At the request of members of the Rock Hill 3 School Board, the administration established a special committee to discuss criteria for determining the impact of the iRock program on student achievement. About 25 people, including parents of students and several teachers, participated in the discussion held on February 7. 2013. The agenda and discussion was led by Professor Marshall Jones of the Winthrop School of Education. Dr. Jones has extensive experience in evaluating the impact of technology on education.

After laying out the problem and a review of the iRock proposal, five subgroups were formed to react to key questions. Three questions were posed for each group to respond to. The transcribed responses of each group are given below that detail issues they saw in need of further planning.

The three areas of focus:

  1. Discuss perceived strengths and weaknesses of the iPad Initiative.
  2. Discuss perceived strengths and weaknesses of the current evaluation goals.
  3. Describe perceived unique Challenges of iPads by Elementary, Middle and High School Level and Data Sources for Evaluating Success.

Information about the five committees reviewing the iRock plan are posted on the district website at http://irockanytimeanywhere.com/the-plan/. That site gives only an overview of the evaluation committee. Below are detailed reactions from each of the five groups at the evaluation meeting, which form a basis for further discussion of needed enhancements for the evaluation plan.

The overview:

Question One: Discuss Strengths and Weaknesses of the iPad Initiative

Group 1 Strengths
1. If implemented correctly, the use of iPads in Challenge Base Learning (projects) better prepares our students for Common Core testing in 2015.
Group 1 Weaknesses
1. Not everything is ready to go….i.e. very few ebooks in school libraries, textbooks are not all digital yet
2. Do we have enough bandwidth to handle 1:1?
3. Already understaffed with Media Specialists and IT people.
Group 2 Strengths
1.Technology is coming and this could put us ahead of the curve. Economic development and jobs would likely result. Chamber should support this.
2.Enable kids to develop their own interests, and engage creatively with material.
3.Good for kids with certain disabilities. One parent has a child with writing difficulty and hopes to use vocal recognition.
4.Textbooks with interactive capabilities would be worlds better. Environmentally better, cheaper we hope, better for kid’s back.

Group 2 Weaknessses
1.One parent has an 8 year old in the iPad cohort who is technological adept. Wish there was more communication on what they are doing with the technology. Not sure what they doing.
2.Parents would like to know what is going on in Challenge Based Learning classes.
3 Online safety is a huge concern. iPads brought home are incredibly vulnerable and parents aren’t equipped to set them up for safety, they need major support to protect their kids.
4. Principals should do a plan for the school in how they are going to use them.
5.Training is on apps, but teachers need to collaborate to do grade level planning to do project based learning. Principals need to lead this or it’s a waste of money. We want to know what the research says about disasters… When achievement doesn’t follow technology, why?
6.Parents need to be an additional source of data!!!
7. We want engagement, and the teacher is the one that matters.
8.How are you going to teach Beowulf creatively with iPads? The focus needs to be on CBL and using the tool.
9.This fundamentally changes the role of the teacher.
10.VERY concerned that this will be used for shallow learning, not deeper CBL.
Group 3 strengths
1. we are all able to go straight to our iPads and find a copy of the irock plan
2. resources are always at our finger tips; dictionaries, encyclopedias, world maps, textbooks, learning games.
3. Research, communication, skype people in other countries, current events,  weather around the world.
4. Keeps kids engaged; students are motivated to use them right now, students can be creative
5. Recording devices, data bases, paperless,
6. No lost books, torn books,
7. Less to carry home, can be used in the car, on the bus
8. There are learning games.
9. Support for Common Core
10. Videos allow students to express themselves and hear themselves.
11. User friendly,   materials don’t get lost…the cloud saves it
12. Instant assessments
13.Working from home if sick
14. iPads become quite personal
15.iPads can be tracked!
Group 3 weaknesses
1. limited number of iPads in the classroom means the memory will fill up too fast
2. teacher training and interest (not all teachers are on board)
3. iPad maintenance is necessary because going from class to class everything has been changed
4. need more tech support…one for each school
5. Cost
6. no wireless at home will be a disadvantage
7. fairness of phasing in…one class and not the others, one team and not the others,
8. still spending money on textbooks
9. teachers being asked to donate money for iPads
10. theft, and breakage
11. iPad training classes need much more time for implementation.
12. iPads do not have flash so laptops will still be necessary
13. some students may forget to charge…need more outlets?
14. learning environments may need to change….iPads are mobile…teachers want to be aware of what the students are doing.
Group 4 strengths
1. tools for students to use and not ways to deliver information – not a glorified Promethian board
2. Ability to choose what you read and how
3. Wealth of access available to student in their hand
4. Student accessing himself
5. Relative easy to use
6. initiating learning in younger & younger students
7. Intuitive device
8. eliminates teachers perceived need to store knowledge and regurgitate
9. what they create and publish it is amazing
10. encourages teamwork and collaboration
11. easy way to get real life/non-fiction text in classroom
12. forces to address quality of service/validity of information
13. excites students about learning
14. Goal of taking teacher to the role of facilitator not deliverer
15. Leadership key in delivering the professional development and modeling use of the digital device
16. Formal and informal feedback from classroom
17. iPad so popular and engaging – cultural status of iPad higher
18. students able to conceptualize problem, develop
Group 4 weaknesses
1. comfort level of teachers
2. use in standardized testing – timeframe to setup – good 15 mins
3. ties to the curriculum – replacing to the all digital environment – ways of assessing – not in current plan – what is body of experience w/ common assessments
4. Common core standards are all application and should be driving this
5. What about teachers that are reluctant to use the device?
6. common core standards requires teachers to use this tool
7. what happens when devices are lost/broken
8. Limitations of device itself
9. Size of memory
10. What about email account for elementary age students?
11. Questions about distribution and monitoring of devices
12. Central delivery of applications of devices
13. Network functions/capacity –schools are not capable of handling

 

Group 5 Strengths (Elementary)
1.student interests
2. Home/school connection- teacher teaches lesson and students can access it from home.
3.There is something in the plan for parents who cannot afford iPads.
4. Works well with investments we have already made (Reflection App, Promethean Board)
5. Prepares students for 21st century/future jobs
6.  Access to internet
7. Can make assessments easier
8.  Good interventions for learners
Group 5 Weaknesses (Elementary)
1. Teachers aren’t prepared to implement the program on a large scale.
2. Not developmentally appropriate for pre-K thru 1st grades.
3. iPad Apps have limited ability to interact with other tablets
4. Parents won’t send them in.
5. Parents may allow their children to download inappropriate apps on their personal device.
6. How do you insure that students are staying on task and not messaging, texting, getting on apps, etc…
7. Exposure to inappropriate images/lack of internet restrictions
8. Lack of fine motor development
9. No paper for sharing
10. Kids without internet at home.
11. Takes away from collaboration if every student has his own device and is expected to work in a group
12. Time consuming to listen to and check students work
13. Concerns that the iPad will become like a workbookQuestion: Is this plan to be like a CBL classroom and use the iPad for the majority of the lesson or is it supposed to be a tool to use when needed?

Question Two: Strengths and Weaknesses of Current Evaluation Goals

Current Goals Proposed by Administration:
1. Student Achievement (Test scores formative and summative, graduation rate, scholarship dollars, size of achievement gap(s), reading ability, etc.)
2. Student Engagement (Gallup Student Engagement Survey matched to demographics, attendance, discipline, dropout rate, etc.)
3. Differentiation (Teacher survey, walk throughs, etc.)
4. Professional Development (Formative and summative teacher and principal surveys)
5. Teacher Perceptions regarding Teaching and Learning (Teacher survey)

Group 1 comments on current goals
1. Fiscal Sustainability? Think this should be added as an evaluation goal.2. All of the current goals need to remain.3. Not sure how we measure the confidence an ESOL student gains, or a social “misfit” gains from using the iPad and becoming a contributing member of the classroom community. Some of these kids are finally fitting in and finding their niche.4. Since there is no control group in our “experiment” we don’t see how to accurately and scientifically measure the success or failure of this program.

Group 2 comments on current goals
Not a consistent list, 3-5 doesn’t jibe with 1-2.
1. Engagement
2. Achievement
3. Security: How will the devices be protected? Are we supporting parents with kid safety?
4. Used to customize education for varying degrees of development.
5. Professional development
6. Surveys (concerns about anonymity)
7. Have we supported parental understanding of the uses of the technology and how to protect their children from online harm?
8. Jargony.
9. This needs to be introduced as a teacher tool to support CBL.
10. Greatest potential for underperforming, disabled, and kids who don’t fit in.
11. Start with alternative schools, failing kids, special ed to pilot program as there might be most bang for buck.
12. There is a need for an evaluation goal to determine the safety of the actual devices…ex: how many devices have been stolen (how many have been traced and recovered), how many have been damaged vs cost of repairs, etc.
13. Must be research based. Best practices. CBL should be priority. Need excellent feedback loops, but don’t reinvent the wheel. Use the experience of other districts.
14. Cultural change, impossible to measure, but essential to success.
15. Focus on deep learning, not shallow learning.
16. Concerns about what we have to give up to get these. RH is weak on field trips, meaningful ones, and technology is no substitute for experiential learning, but I still want it for CBL learning as a tool, part of the package. Not the point, or the final point, but as part of the process to engagement.
17. Evaluation of how teachers use the technology well. Encourage and monitor teachers, peer reviews.

Group 3 comments on current goals
1. Parent and community perception regarding teaching and learning on the iPads
(Parenting in-services)(Parents come in to learn with or from their own children)
2. iPad management (wireless access at school and at home)( location of iPad carts)( charging iPads each night) (security) (if sharing…fair usage, sign up sheets) (filtering)
3. Under” teacher perception” (create a technology competency test and required in-services until learned.)( teaching to the teacher) This needs to be a requirement
4. Differentiation. Students can work on their ability level. Track student work. Students can speak their answers instead of writing. Spell check on the iPads. Typing instead of writing.
5. ongoing professional development bring in substitutes for more time

 

Group 4  Comments on Current Goals:
1. Student Achievement (Test scores formative and summative, graduation rate, scholarship dollars, size of achievement gap(s), reading ability, etc.)
   Group Comments
     1.RHSD accountability measures – implementation gap because there is skill  gap.
2. Three years to fully implement in elementary and 6 years in secondary?
When do you expect to see results? Where is teacher evaluation in picture?
Where are you evaluating quality of what teacher is delivering?
Defining success and achievement? Does this measure what we value?

Current Goal 2. Student Engagement (Gallup Student Engagement Survey
matched to demographics, attendance, discipline, dropout rate, etc.)
Group 4 comments
1.Engagement influences achievement and increases consistent attendance; greater engagement decreases discipline issues; does greater engagement cause students to take more challenging classes.
2. Would engagement increase so students were taking higher level classes – no more early release. Look at school to work and licenses and certificates. Dropout rate decrease and possible impact to ATC enrollments. Are we promoting school to work initiative in SC. Outlet for skills

Current Goal 3  Differentiation (Teacher survey, walk throughs, etc.)
Group 4 Comments
1.What is the goal of differentiation, is there differentiation, what is the evidence that differentiation is a function of the iRock initiative? Able to differentiate curriculum w/o having to create individually?

Current Goal 4. Professional Development (Formative and summative teacher and principal surveys)
Group 4 Comments
1.Keep this goal – it forces district to invest in professional development. Teachers have to be able to troubleshoot. We do not do formative surveys and limited summative . What prompts professional development now is student achievement test scores OR perceptions of student achievement test scores.
2. In learning/evaluation plan professional development is infused throughout. Must log PD hours into My Learning Plan for annual evaluation.

Current Goal 5. Teacher Perceptions re: Teaching and Learning (Teacher survey)
Group 4 Comments
1.  Do you want to measure changes in teacher perceptions about learning due to iPads. Are they focusing less and more on the student learning?
2. Dynamics of ways teachers treat each other – value of new faculty will be more important.
3. Self critique and evaluation important.  When you are forced to learn something new, different stages – frustration,

Other Group 4 Comments:
1. U.S. strength is to be creative.
2. No standardized approach to teachers that were given iPads, no standardized curriculum; not deployed in same way at different schools; presented under an idealistic umbrella that was not confirmed to be what had happened prior. Establish 3 years from now limited variables and consistency across elementary. First base year would be 2013-14; some schools allowing them to take them home others just beginning process; some sharing in groups.

Group 5 Elementary Current Goals

Goal 1 Student Achievement
1. How will we know if the iPad initiative is the thing that changed the data OR our shift from SC State Standards to Common Core Standards?
2. How will we compare students’ test scores from the current PASS test to the future Smarter Balance Test?
3. What will the evaluation be compared to? Year to year? If everyone gets an iPad, who will they be compared to, since we have them this year and we will be going 1 to 1 next year?
4. If everyone gets them at once, who/what is the control group?
5. Since technology usage does not necessarily increase test scores, is this the best path to take? Would it be more beneficial to decrease class sizes and hire more teachers with the money?
Goal 2 Students Engagement
1. Other evaluations should be looked at to determine the benefits of iPad usage.
2. Employment rate after high school?
3. With elementary kids, the iPad is not necessarily an attendance issue.
4.  Track home usage of iPads at home
Goal 3 Differentiation
1. Concern: Some teachers think that differentiation is giving students different leveled apps.
2. The number of kids brought before RTI
Goal 4: Professional Development
1. The number of voluntary professional development courses/hours related to iPads.
Goal 5  Teacher perceptions
1. Add parent surveys

Question 3:  Unique Challenges of iPads by Elementary, Middle and High School Level and Data Sources for Evaluating Success.

Group 1  Challenges for Middle Schools
1. Puberty- kids are the most varied physically in middle school
2. protection at home and school security is a necessity
3. parents need to know what the kids are capable of and how to protect them from all that is “out there”
4. Is there a cellular cut off for certain apps?
5. adopting common core at the same time as the iPad may put a bad light on the iPad because scores on common core will probably be lower
6. parent involvement middle school parents—kids don’t want their parents around…
7. lack of understanding in the community financially and instructionally
8. lack of teacher training, resources,

Group 1  Data Sources for Middle School
1. middle school parent surveys, parent perceptions, understanding, classes for parents.  Create a focal point for parent night or rap sessions
2. MAP
3.Pass
4. Compass
5. IEP
6.Resources

Group 2 Unique challenges for High School.
1. Already a lot of conflict stems from social media – different medium; invasion of privacy to video. May be encouraged in one course and discouraged in another class. Criticism no device management for download of apps.
2.Resources allocated to manage protection/acquisition for what is downloaded to device – in district managed system can control every app on device vs student managed system can go anywhere and download anything they want. District 3 made choice to do it before they sold idea – down low budget – issues related to phase in.
3.Challenge is consistent implement across one school. In high school there is more of a importance on individuality on a teachers style, craft than at other levels. Encourage teachers to take risk and practice.
4. Degree of expertise faculty feel in field is stronger than evaluator.
5.We are going to get a device in every person’s hand and change outcomes? Point not that everyone has an iPad but how it will help our students. Technology is a given, we are going to technology, how do you make sure you have done it well and that our students have learned. When you go to measure something it has to relative to surroundings.
6.  This conversation should have happened a year ago.
7. Changing community’s focus from iPad to technology

Group 2 List of data sources unique to HS.
1. MAPP scores exist for 9th grade, needed policy change district wide.
2. EOC and HSAP
3. ACT future assessments
4. Course specific benchmark tests
5. Could work on district wide assessment


Group 3 Elementary Unique Challenges
1 Teachers aren’t prepared to implement the program on a large scale.
2. Parent buy in- parents who are concerned about their child being responsible enough to carry it back and forth from school.
3.How will teachers be able to monitor students staying on task/ classroom management.
4. The amount of screen time
5. Add technology screening to pre-screen
6. Parents with more than one child in the district- finances

Group 3 Elementary Data Sources
1. Fontas &Pinell
2. CogAt
3. MAPS
4. PASS/Smarter Balance
5. Success Maker

 

 

This entry was posted in 21st century schools. Bookmark the permalink.

Comments are closed.