content: its-science
This data as json
| author | category | content | published_date | slug | summary | title | url |
|---|---|---|---|---|---|---|---|
| ryan | musings | I have a 10 year old daughter in the fifth grade. She has participated in the Science Fair almost every year, but this year was different. This year was required participation. dun … dun … dun … She and her friend had a really interesting idea on what to do. They wanted to ask the question, **“Is Soap and Water the Best Cleaning Method?** ” The two Scientists decided that they would test how well the following cleaning agents cleaned a white t-shirt (my white t-shirt actually) after it got dirty: * Plain Water * Soap and Water * Milk * Almond Milk While working with them we experimented on how to make the process as scientific as possible. Our first attempt was to just take a picture of the Clean shirt, cut the shirt up and get it dirty. Then we’d try each cleaning agent to see how it went. It did not go well. It was immediately apparent that there would be no way to test the various cleaning methods efficacy. No problem. In our second trial we decided to approach it more scientifically. We would draw 12 equally sized squares on the shirt and take a picture:  We needed 12 squares because we had 4 cleaning methods and 3 trials that needed to be performed > > 4 Cleaning Methods X 3 Trials = 12 Samples Next, the Scientists would get the shirt dirty. We then cut out the squares so that we could test cleaning the samples. Here’s an outline of what the Scientists did to test their hypothesis: 1. Take a picture of each piece BEFORE they get dirty 2. Get each sample dirty 3. Take a picture of each dirty sample 4. Clean each sample 5. Take a picture of each cleaned sample 6. Repeat for each trial For the ‘Clean Each Sample’ step they placed 1/3 of a cup of the cleaning solution into a small Tupperware tub that could be sealed and _vigorously_ shook for 5 minutes. They had some tired arms at the end. Once we had performed the experiment we our **raw** data: # Trial 1 Method Start Dirty Cleaned * * * Water    Soap And Water    Milk    Almond Milk    * * * # Trial 2 Method Start Dirty Cleaned * * * Water    Soap And Water    Milk    Almond Milk    * * * # Trial 3 Method Start Dirty Cleaned * * * Water    Soap And Water    Milk    Almond Milk    This is great and all, but now what? We can’t really use subjective measures to determine cleanliness and call it science! My daughter and her friend aren’t coders, but I did explain to them that we needed a more scientific way to determine cleanliness. I suggested that we use `python` to examine the image and determine the brightness of the image. We could then use some math to compare the brightness. 1 Now, onto the code! OK, let’s import some libraries: from PIL import Image, ImageStat import math import glob import pandas as pd import matplotlib.pyplot as plt There are 2 functions to determine `brightness` that I found [here](http://stackoverflow.com/questions/3490727/what-are-some-methods-to- analyze-image-brightness-using-python). They were super useful for this project. As an aside, I love StackOverflow! #Convert image to greyscale, return average pixel brightness. def brightness01( im_file ): im = Image.open(im_file).convert('L') stat = ImageStat.Stat(im) return stat.mean[0] #Convert image to greyscale, return RMS pixel brightness. def brightness02( im_file ): im = Image.open(im_file).convert('L') stat = ImageStat.Stat(im) return stat.rms[0] The next block of code takes the images and processes them to get the return the brightness levels (both of them) and return them to a `DataFrame` to be used to write to a `csv` file. I named the files in such a way so that I could automate this. It was a bit tedious (and I did have the scientists help) but they were struggling to understand why we were doing what we were doing. Turns out teaching CS concepts is harder than it looks. f = [] img_brightness01 = [] img_brightness02 = [] trial = [] state = [] method = [] for filename in glob.glob('/Users/Ryan/Dropbox/Abby/Science project 2016/cropped images/**/*', recursive=True): f.append(filename.split('/')[-1]) img_brightness01.append(round(brightness01(filename),0)) img_brightness02.append(round(brightness02(filename),0)) for part in f: trial.append(part.split('_')[0]) state.append(part.split('_')[1]) method.append(part.split('_')[2].replace('.png', '').replace('.jpg','')) dic = {'TrialNumber': trial, 'SampleState': state, 'CleaningMethod': method, 'BrightnessLevel01': img_brightness01, 'BrightnessLevel02': img_brightness02} results = pd.DataFrame(dic) I’m writing the output to a `csv` file here so that the scientist will have their data to make their graphs. This is where my help with them ended. #write to a csv file results.to_csv('/Users/Ryan/Dropbox/Abby/Science project 2016/results.csv') Something I wanted to do though was to see what our options were in `python` for creating graphs. Part of the reason this wasn’t included with the science project is that we were on a time crunch and it was easier for the Scientists to use [Google Docs](https://docs.google.com) to create their charts, and part of it was that I didn’t want to cheat them out of creating the charts on their own. There is a formula below to determine a `score` which is given by a normalized percentage that was used by them, but the graphing portion below I did after the project was turned in. Let’s get the setup out of the way: #Create Bar Charts trials = ['Trial1','Trial2','Trial3'] n_trials = len(trials) index = np.arange(n_trials) bar_width = 0.25 bar_buffer = 0.05 opacity = 0.4 graph_color = ['b', 'r', 'g', 'k'] methods = ['Water', 'SoapAndWater', 'Milk', 'AlmondMilk'] graph_data = [] Now, let’s loop through each cleaning method and generate a list of scores (where one score is for one trial) for singlemethod in methods: score= [] for trialnumber in trials: s = results.loc[results['CleaningMethod'] == singlemethod].loc[results['TrialNumber'] == trialnumber].loc[results['SampleState'] == 'Start'][['BrightnessLevel01']] s = list(s.values.flatten())[0] d = results.loc[results['CleaningMethod'] == singlemethod].loc[results['TrialNumber'] == trialnumber].loc[results['SampleState'] == 'Dirty'][['BrightnessLevel01']] d = list(d.values.flatten())[0] c = results.loc[results['CleaningMethod'] == singlemethod].loc[results['TrialNumber'] == trialnumber].loc[results['SampleState'] == 'Clean'][['BrightnessLevel01']] c = list(c.values.flatten())[0] scorepct = float((c-d) / (s - d)) score.append(scorepct) graph_data.append(score) This last section was what stumped me for the longest time. I had such a mental block converting from iterating over items in a list to item counts of a list. After much [Googling](https://www.google.com) I was finally able to make the breakthrough I needed and found the idea of looping through a range and everything came together: for i in range(0, len(graph_data)): plt.bar(index+ (bar_width)*i, graph_data[i], bar_width-.05, alpha=opacity,color=graph_color[i],label=methods[i]) plt.xlabel('Trial Number') plt.axvline(x=i-.025, color='k', linestyle='--') plt.xticks(index+bar_width*2, trials) plt.yticks((-1,-.75, -.5, -.25, 0,0.25, 0.5, 0.75, 1)) plt.ylabel('Brightness Percent Score') plt.title('Comparative Brightness Scores') plt.legend(loc=3) The final output of this code gives:  From the graph you can see the results are … inconclusive. I’m not sure what the heck happened in Trial 3 but the Scientists were able to make the samples dirtier. Ignoring `Trial 3` there is no clear winner in either `Trial 1` or `Trial 2`. I think it would have been interesting to have 30 - 45 trials and tested this with a some statistics, but that’s just me wanting to show something to be statistically valid. I think the best part of all of this was the time I got to spend with my daughter and the thinking through the experiment. I think she and her friend learned a bit more about the scientific method (and hey, isn’t that what this type of thing is all about?). I was also really excited when her friend said, “Science is pretty cool” and then had a big smile on her face. They didn’t go onto district, or get a blue ribbon, but they won in that they learned how neat science can be. 1. [The score is the ratio of how clean the cleaning method was able to get the sample compared to where it started, i.e. the ratio of the difference of the `cleaned` sample and the `dirty` sample to the difference of the `starting` sample and the `dirty` sample. ↩︎ | 2016-12-17 | its-science | I have a 10 year old daughter in the fifth grade. She has participated in the Science Fair almost every year, but this year was different. This year was required participation. dun … dun … dun … She and her friend had a really interesting idea on what to do. They wanted to … | It's Science! | https://www.ryancheley.com/2016/12/17/its-science/ |