Fundamentals of Science: Observation, Replication and Prediction

Steve Trimberger
The Trimberger Family Foundation

We are going to talk about Science, but this is probably a different view of Science than the one you've been studying. To understand this different view, let's start by talking about language studies. In language, you study spelling and grammar. These are some facts of language. You need them to write well, but there is more to being a writer than being able to spell correctly and use correct grammar. So it is with Science. Much of what you've learned about Science are the facts of Science. Today, we'll investigate Science in a little more detail. And just like in language, you need to be able to read and write well even if you don't eventually become a writer; so it is with Science: you will be well served to understand Science well, because it can help you even if you don't grow up to be a scientist.
We are going to talk today about what scientists do, how they work. And how Science works. We will talk about a way of thinking about problems. We will touch on various problems as we go along. We'll do some experiments and we'll talk about not only what we discover, but how we discover it.
Let's start by asking "What is Science?" <pause for reply from class>

Science is the study of nature. We want to find out how nature works. We want new information.
So think about this: If you are a scientist, and you are trying to figure out something brand new, how do you know when you've got the right answer? <pause for effect> There's no teacher around to tell you. <pause for effect again> Let's talk about that.

First, if you are trying to discover something new, you need to be able to record, to notice, exactly what happens. Because you don't know what is really going to happen. In some ways, this is very easy: you just need to look and see what it is. In some ways, this is very hard, because you might have a preconceived idea of what should happen, so you are really looking for what you think should happen rather that what really happens. This is a scientist's most important job: to be a good observer.

OK, so you've observed something. How do you know it is right? How do you know you observed it correctly? Or wrote it down correctly? Or there wasn't something else happening that you didn't notice (like someone bumped the table or something) that messed up your observation? Why, you just do it again. This is the second step: replication. If we observe something, and we describe what we did and other people can repeat our experiment and observe the same result, then we can conclude we've correctly observed what is true.

This is a big deal and it is the reason why scientists don't care much about reports of ESP (Extra-Sensory Perception): mind reading and stuff like that. Not because they don't like the people doing it, but because other people are unable to repeat the experiment and get the same result. It's that simple. In science, if you can't replicate it, then you don't understand it. It isn't science. Accidents and coincidences happen all the time. Nobel Prize-winner Richard Feynman said science is what we have learned about how to keep from fooling ourselves.

Once we can replicate an observation, we can can propose a mechanism in nature that explains the observation (scientists call this kind of proposal a theory), or we can devise a mathematical relationship between parts of nature (a law). For our theory or law to be part of science, it must be able to predict a result of an experiment that has not yet been done. If it can do this more successfully than other theories, then it will be accepted. This is also important. If your explanation only explains what's already been observed, then that's OK, but not very convincing. Anyone can come up with an explanation for things they see. The really good ones explain what no one has seen yet.

So, these are three vital parts of science that we are discussing today: observation, replication and prediction.

The Experiment

OK, so now we'll do our experiment. In our experiment, you must practice being a good observer. Remember to record what you see, regardless of what you expect to see. A good observer will also notice aspects of the experiment, the experimental setup and exactly what they did and what happened during the experiment.

We will break up into several groups (note: at least 8), so we will have the experiment replicated, too. In this experiment we will time a pendulum. I have stands made from K-nex, strings of various length and different size nuts and washers to hang from the string. I also have stop-watches that you will use to time the pendulum for ten oscillations.

Now run the experiment and write down your result. When you are done, please don't disturb anything about your setup.

During the Experiment

<Teachers: During the experiment, answer most questions with "use your best judgment and observe". Most of the answers won't affect the results materially, and the questions are usually raised by the mind-set that wants to get the "right" answer rather than observe what they do. Typical questions include: "How do I release the weight?", "How do I keep it from hitting the side?" "What if it goes around in a circle?"
One question that does affect the result is "Is an oscillation one swing or over-and-back?" This one DOES affect the result, but I don't clarify it anyway because later, when we graph the results, we'll identify the two different understandings of the meaning of "oscillation" as a source of error and it usually helps the messy data become clear rather dramatically.>

Some students finish quickly, others take longer. Some will take a very long time. Some never complete. After about five minutes, nearly everyone should be finished.

Finishing quickly is not important. Having everyone finish is not important for science. But, only the very first scientist to report results gets the Nobel Prize. So finish up! <If one group just can't finish, and the class is antsy, go on "This is why we replicate the experiment. No one group can hold up the path of science.">

Experimental Results

Now lets look at the results. When I did this experiment, I got 6 seconds, who agrees with me? Who got about six seconds? Raise your hand. <pause> That settles it, six seconds is the right answer, right? Wrong! That has nothing to do with science. Being a good observer means recording your results regardless of what you thought was the right answer and regardless of what some authority (like me or your teacher) says it is supposed to be. You must record what you observed. Besides, who is to say what "about six seconds" means. So let's get the times you observed. The times below are typical for a class:

Group	Time
1	7.82
2	5.97
3	4.47
4	8.37
5	3.36
6	5.91
7	3.11
8	13.65

Let's see if we all agree on an answer. Well, there are a few around 6, 7, and 8, and also a few in the threes and some in-between, and one that disagrees with everyone else. We can't really say there's agreement here. This brings up the question: why not?

Sources of Error

Scientists talk about "Sources of Error" in experiments. In this case, "error" usually doesn't mean "mistake", but just means some reason why the results don't match some pre-conceived notion (like different people doing the same experiment ought to get the same answer). The source of error is important, because it may lead to new, better understanding of the world -- truly new knowledge. That's the road to a Nobel Prize. If we really want to go after new knowledge, then we look for the greatest disagreement and check that experiment to see what's different. In this example, we ask group 8 why theirs might be different. They are closest to a Nobel Prize right now!

So now, I'll ask you, what are possible sources of error in this experiment? Why would different scientists get a different answers? Let's start by asking the group that is closest to the Nobel Prize, Group 8. Then we'll ask everyone. <pause and collect answers>

<Teachers: Here is a list of typical responses:

· Different height of the stand
· Different ways of measuring
· Weight went around in a circle instead of back and forth
· Weight hit the side
· How high the weight was when you released it
· Length of string
· Mass of weight
· Measured something else (note: not "the wrong thing" nothing is "wrong") (miscounted, etc.)
· Watch didn't work right (problems with equipment)

So, let's try something easy. The stands that hold the pendulum differ slightly. Some are taller, some wider. The K-nex pieces that make up the stand are different color for different length. So, each group, please report the color of the K-nex piece that forms the leg of your stand. We'll see if that makes a difference.

Group	Time	Color
1	7.82	Grey
2	5.97	Red
3	4.47	Grey
4	8.37	Grey
5	3.36	Red
6	5.91	Grey
7	3.11	Grey
8	13.65	Grey

Well, just standing here, I don't see a pattern. The red ones might be with smaller numbers, but we have both the smallest and largest with grey stand pieces.

Further Investigation - Measuring the Strings

Let's not be swayed, though. Let's try another source of error. How about string length? Please measure your string length and we'll add that to the chart.

Group	Time	Color	String_Length_(cm)
1	7.82	Grey	14.7
2	5.97	Red	8.6
3	4.47	Grey	4.8
4	8.37	Grey	16.5
5	3.36	Red	11.6
6	5.91	Grey	8.4
7	3.11	Grey	13.3
8	13.65	Grey	8.6

Plotting the Data

I still don't see much of a pattern. Let's plot it and see what it looks like:

At first glance, I don't see anything going on with string length, either, but if you ignore some of the results maybe there is a pattern. We seem to have a bunch on a line, and #5 and #7 clustered off it, and #8 still way off in space, looking for a Nobel Prize. There's still nothing here that looks like a good pattern. Let me ask now, "What is an oscillation?" There was some question about this when we did the original experiment. Who decided one oscillation is over-and-back, and who decided each one-way swing of the pendulum is an oscillation? <pause to collect responses> There's no correct or incorrect response. We just want to be sure we are all measuring the same thing. <At this point, I found groups #5 and #7 chose each one-way swing was one "oscillation", the others all decided over-and-back was one oscillation.>
Those that measured each one-way swing as an “oscillation” would count five over-and-backs, so groups would get a number much smaller than the others (probably about half). To make a conclusion, we'll just skip over those two. <Teachers: if most chose one-way as an “oscillation, then skip over the over-and-back ones> We'll ignore them for now because they measured something else.

But what about #8? We can ask that group what was different, but in Science, we'll often look at the preponderance of data (everybody except #8, now) and just say we don't know why #8 was different. We don't need to know. There was something funny about that one and we'll make our claim that there is a relationship between the length of the string and the time of the oscillations. We can develop a law and make a prediction: we can draw a line through the points on the graph that will predict the times for much longer and shorter strings.
So, giving this a quick eyeball guess, I'd say we can write a law that says given the length L of the string in centimeters, ten oscillations should take time (T):

T = 3 + 0.325 L

This explains our data and predicts future experiments. Is it “correct”? In science, "correct" means it explains the world. This explains the world we observed. If you want to see if a professional physicist agrees, go ahead and ask.
<Teachers: sometimes at the end of this process, there is still no pattern, then you must conclude that there is no relationship between length of string and string length (which disagrees with what the physics books tell us). But, importantly, if your data doesn't support any other conclusion, you can't claim one. It's OK, though, just say “We found no pattern here. Now, physics books may disagree with us, and if you think about it, a really long string ought to be much slower. But right here and now, we don't see it. Perhaps we have observed a new phenomenon in Science. Or maybe we just make a mistake. It doesn't matter. We are good observers, so we will report what we found. This is the essense of today's lesson.”>

The Need for Replication

This lesson points out the need for replication for the advancement of understanding. A single point, or one that could not be repeated, is useless. Even two or three might not be enough. But when we've got several, then we can see which data points agree and which do not. We can choose to investigate the sources of error, as we did originally with the string length. And we can choose to ignore some data, like #8, because they doesn't agree with all the other results we have.

But What about the Outliers?

We've dealt pretty harshly with numbers we don't agree with -- we ignored them without trying to prove them wrong. Is that fair? No, not really. It often happens that not all numbers agree. Even those we included along our line don't fall exactly on the line. So what happens in Science? Well, first off, when there is one observation that doesn't agree with the others, and there are lots of others, then the burden of proof is on the person who wants the new information believed. If group 8 wants their data to be believed, they need to explain the source of error, and collect additional data that overwhelms the results we have so far. The burden of proof is on group #8.

This is not bad. Remember, Nobel Prizes are given to those who discover something important, and the best way to find that important something is to look at the errors in old experiments, to find out what everyone else missed in their rush to write a scientific law. The famous scientist Albert Einstein explained his theory of relativity, and many people thought it was too strange to be real. His opponents even wrote a pamphlet titled "100 Authors Against Einstein". Einstein responded "If I were wrong, one would be enough." It is true. One good piece of data will doom the best theory. It may take time, but it does happen. It has happened over and over again in Science.

Some Concluding Thoughts

We made trouble for ourselves because we didn't agree on terms, like what is an "oscillation". Plus, everyone's equipment was different. In practice, scientists are very careful about their terms. That's why, for instance they use "mass" instead of "weight". There is a difference. In practice, also, scientists try to duplicate experiments exactly before trying to figure out what to do next. And an individual scientist will run the same experiment several times, just to make sure nothing really strange happened one one try. That's replication. Some of you may have done the same, timing the pendulum several times. Good job!

As I said, scientists try to "control the variables" by making as much as possible identical between replications of an experiment, then varying one thing at a time to understand it. Think about timing your pendulum ten times, then changing the string length and timing ten times for each of ten different string lengths, then ten times for each of ten different weights at different string lengths. Boring? Perhaps. But think about how you felt when you saw the numbers start to line up on our graph. There is order in the universe, and this is the way we know to discover that order.

Some Examples of Science in Action

Just as an aside, did you ever wonder how scientists decide what to investigate? In a whole universe full of things to look at, why choose what they choose? Part of it is what interests them. Imagine that! Getting paid for finding out about things that interest you! But then, where to look? One method is to look at previous experiments, previous knowledge, and observe very carefully, measure very precisely, and look for "error", where the results don't match the best scientific explanation of the day. Then investigate the potential "sources of error" and understand better what has been going on.

Another way is to observe something that has never before been observed carefully. This can get very confusing when there are lots of observations, but no theories or laws to explain them. This is where we are with much of biology and genetics today: many observations, but theories don't explain the observations very well, and they don't make good predictions. How do genes really work? How does human memory work? Exactly what is aging: why do we get old? Why does a fifteen-year-old heal fast and get stronger every year, but an eighty-year-old gets weaker every year and can die from a broken bone? Someday, someone will come up with some good theories in this area, become very famous and win a Nobel Prize. Maybe it will be you.

Galileo's Gravity Experiments

I'll give two examples of Science in action. One is very old, one is very new. Five hundred years ago, Galileo Galilei was interested in falling objects. People of his day said that heavy things fall faster than light things (a bowling ball falls faster than a feather). This makes some kind of sense, but Galileo didn't quite believe it. Think about a landslide or rocks falling off a cliff. Do the big ones fall faster than the small ones? No. Or at least not enough faster to be observable. So Galileo made ramps and rolled balls down them (so everything happened slower). It is hard to time rocks falling off tall cliffs, not to mention dangerous and tiring to keep carrying them back up. He discovered that the weight didn't matter, and he discovered more: a mathematical law that described how gravity worked. It predicted how fast an object would travel after falling for a given amount of time. Galileo's measurements were the basis of understanding gravity.

Solar Neutrinos

My second example is very new. In fact, the story is not done yet. Here's how it goes. If I were to ask you if you know what makes the sun hot, maybe you could give an answer. Maybe you know, maybe not (it's a huge nuclear reaction). If I ask a professional Physicist, a scientist who is an expert in what makes stars hot, he might say he knows. But if I were to ask a nuclear Physicist, someone who knows exactly how atoms behave, she would claim to not really know. And here's why.
The sun is hot because of nuclear fusion -- atoms of hydrogen merging into helium. That releases lots of heat. OK, but if you look very carefully at what must happen, and scientists have, you discover that during that process some tiny particles are released. These particles are called neutrinos. They are extremely tiny and extremely hard to find because they are so small. For years, they could not be observed, but they had to be there (sort of like the spokes on a bike that's going fast -- you can't see them, but if they weren't there then nothing else makes sense either). Well, Raymond Davis decided to check -- to look for neutrinos coming out of the sun. He and his group built a huge detector in a mine in South Dakota and observed ... not enough neutrinos from the sun. This was a shock, because if there weren't enough neutrinos, then either our understanding of basic physics was wrong or the sun has shut down (partially), which would mean we're all about to freeze to death. Well, there was another explanation: maybe there was an error. After all, the experiment had not been .... ? <pause for answer: replicated> So, before panicking, the experiment was reproduced, by Masatoshi Koshiba with a different detector (so just in case there was a source of error in Davis's equipment, it wouldn't be a problem, Koshiba's experiment in Japan was called Kamiokande. And his answer was ... the same! So, either we are all going to freeze to death because the sun has shut down, or our best scientists don't really understand the basic nuclear reactions that power the sun (and hydrogen bombs), or we really don't understand the universe very well.

Right now, physicists are working on a new understanding of these basic reactions. And the new best understanding is that neutrinos have a tiny, little mass, which contradicts the best theories we have of how the universe is put together. Not only that, but if neutrinos have a tiny mass, there are so many of them in the universe that they outweigh all the planets, stars and galaxies! Davis and Koshiba won the Nobel Prize in 2002 for their discovery. Well, for their observation. Just for observing very carefully!

About “Accidental” Discoveries

You often read about a scientist making an "accidental" discovery, for example an astronomer may find a new moon around Saturn. When asked if she was looking for a new moon of Saturn, she would probably say no. She was just looking, just observing. We might say this was accidental: she found the moon without looking for it. But that astronomer has been just looking for anything, without being biased, for fifteen years. Is it really an accident? Davis and Koshiba were just looking for neutrinos. They weren't trying to discover a new understanding of the sun, but they did. Saying it was accidental is like saying it was an accident that you found a quarter in the sand on the beach after you were looking in the sand all afternoon for interesting stuff. Keep looking. Who knows what you'll find!