It's What You Do With It

https://twitter.com/#!/science_goddess/status/198786771260162049

If I asked you how big your data set was, would you hold it against me?

Most days, I feel like what's happening in the world of data is a lot like biological evolution. Outside pressures can drive internal changes. I see a Cambrian-levels of explosive growth in data use and modeling today as we experiment with what outside tools can do to internal data sets. And "Big Data" is rapidly becoming the king beast on the block. Even education cannot escape this growth, with federally funded longitudinal data systems and the other ways districts and states are putting together their data.

https://twitter.com/#!/gnat/status/191606855666118658

But this blog is about small data. I want individual teachers and principals to be able to answer their own questions---without fancy tools, mind-bending formulas/programming, or worrisome charts. Small data has its challenges, however. Statistically, we are on shaky ground. We haven't kicked it around much here, but I do wonder about the validity of the instructional decisions we make based on an assessment of a class of 20 - 30 students. I realize that classroom-level data serves a very different purpose than #BigData. As teachers, we try to support each student. But what represents a reasonable "sample size" for making these determinations? What do validity and reliability look like within the microcosm of small data?

In spite of these (and other) questions, I believe that small data has a significant role within the data ecosystem. We just need to help more people see what that is. I worry that #BigData is on a path to be too large and in charge of what happens in a classroom between an individual teacher and a student. We need to remember that it isn't the size of the data set that matters, it's what you do with it.

Bonus Round
If you live in the Seattle area and want to talk about all sizes and shapes of data, I hope you will join the Seattle Data Visualization Group. The next meetup will be Tuesday, May 22, venue TBD (but likely close to the Tableau folks). Don't worry about the size of your data set---you'll find lots of friendly people who have interesting projects and ideas to share. There were only two of us ed folks there last time, and being able to share our challenges and applications made for great conversation. I'd love to see you there!