In the social sciences we try to understand people: what drives them and how they relate to one another. And I want to start with something simple. Each of us is a mixture, a blend of a great many traits and of a great many things that have happened to us in life.
Pick one variable, say happiness. How happy are you with your life? Your answer depends on a great many things: on genetics (and on a lot of genetics), on how your parents cared for you, on the people you met, on the things you achieved, on the place where you were born, and of course on many more things, some of which are hard even to think of.
And where is luck? It is not just one more factor on the list. It sits above, over everything. Luck and effort together decide where we were born, which parents we got, and whether we grew up with a secure or an anxious attachment style. From there they work through our traits. Some of these traits we know about and know how to measure. Others we have not thought of yet, or have not measured yet. In the language of research, happiness is the dependent variable, and all the traits are the independent variables that predict it. Whatever the traits we measured fail to predict is called "error". But it is not really an error: it is the things we have not measured yet, or measured without enough precision. We will meet these ideas again later in the book, when we talk about the reliability of measurement and about prediction error in regression.
You can think of it as a ball falling through the Galton board (also called a quincunx), which you can see below. The board has rows of nails. Every time the ball hits a nail, it has to "decide" whether to fall to the left or to the right. Each row of nails stands for one layer of life: the first row is your genes, the second your parents, the third your friends, and so on. At every new layer the ball meets another influence on our happiness, and that influence nudges it a little toward more happiness, or a little toward less.
Of course, this is a simplification. In life it is not a simple yes or no, and some things have a big effect while others have a small one: whether your neighbor smiled at you this morning will affect your happiness a little, while good parents affect it a lot. But to keep things simple, let's assume that at every layer there are only two options: right, a little more happiness, and left, a little less.
What are the chances that everything falls on the side of low happiness: less favorable genes, parents who did not take care of you, and friends who were not good friends? Sometimes, sadly, these things do come together. But the chance of falling on the side of low happiness at every single layer, or on the side of high happiness at every single layer, is very small. Chance brings most of us somewhere to the middle.
And we humans notice mainly the layers where we fell on the side of low happiness, and we say things like "Why does this always happen to me?" Bad things stand out for us much more than good things. The layers where we fell on the side of high happiness we hardly see at all, because they seem obvious to us.
No matter what, our eyes are drawn there, and that is sad. It is worth looking at the good things too, and then we will see that somehow they balance each other out. And it is important to remember that not everything is fixed in advance. We can change our point of view, and many of these layers are in our own hands.
Before we play, let's follow a single ball, layer by layer. At the first layer, genetics: it can raise our happiness, or lower it. At the second layer, our parents: they can be better or not as good, and they too push the ball toward more or less happiness. At the third layer, our friends, and again: up or down. And so on, until the ball lands at the bottom.
Now play with the board. Each ball is a person: up, down, up, down, and in the end it lands somewhere. Most balls end up in the center, some to the sides, and very few reach the far edges, a very low or a very high level of happiness. Things balance out. And why? Because there are many more paths leading to the central areas than paths leading to the edges. On the board you can see next to each nail how many paths reach it, and discover that far more paths lead to the middle than to the sides. The shape that forms is, in many cases, a normal distribution.
Drop just a few balls, and the shape will not be neat. The more balls you drop, the more the natural tendency of the board shows itself: the shape you would get with an infinite number of balls. The more people there are, the better we can see how all these factors affect the population as a whole. Many people in the middle, and few with extreme happiness, very low or very high.
Until now we assumed that at every layer the chances are fifty-fifty. But suppose there are people, or whole populations, in whom the pull toward high happiness is stronger: at every layer there is a high chance of moving up in happiness, say 80%, and only 20% of moving down. That goes for genetics, for parents, for friends. On the board, raise the chance that the coin lands on the happiness side, and see what happens: most balls land at a high level of happiness, the distribution becomes skewed, and only a few remain in the long tail on the side of low happiness. It can also work the other way: a population in which the pull toward low happiness is stronger.
This is what statistics is built on: probabilities. Have you met someone with an especially high level of happiness? We can calculate the chance of reaching that corner when every layer is fifty-fifty, and see that it is very, very small. When something is that rare, we ask: maybe this person does not belong to this population at all? Hypothesis testing answers that question, and we will get to it later in the book.
Two more things. The first is the position of the triangle's top. Let's say this is Earth. On Mars the top of the triangle sits further to the right, in the region of higher happiness, and Martians are happier on average. They differ among themselves too, and most of them are happier than the people of Earth. Most, not all: the two distributions overlap. And whoever is at the edge can be "a tail among lions" or "a head among foxes" (as the saying goes, better a tail among lions than a head among foxes): the least happy Martian can be less happy than the happiest person on Earth. This is how we can compare populations. The second is the base of the triangle: some triangles have a very wide base, and some have a very narrow one. In other words, the angles at the bottom of the triangle decide whether the people in a population will differ from one another by a little or by a lot. Later on we will call this large variance or small variance.
It is all here: the mean, the spread, the shape, probability, and comparing populations. This is the foundation of statistics. And now it is your turn to play.
Part 0: Before we start · Introduction
How to read this book
This book was written for people who have never studied statistics, and also for people who studied it and forgot. It starts with school arithmetic, and goes all the way to the analyses you find in research papers in education. This short chapter explains how the book is built, and how to get the most out of it.
Can you learn statistics without being afraid of it?
Yes. Most of what makes statistics hard is not the arithmetic, but the words and the symbols. So every idea in the book starts with a familiar question, is explained in simple words, and only then gets a symbol and a formula. And if something is not clear, that is a sign it needs a better explanation, not a sign that something is wrong with you.
Three choices, in the bar at the top
At the top of every page there are three choices. They are saved in your browser, so you choose once and the whole book adjusts itself.
Language
The book is available in several languages, with more to come. It was written first in Hebrew, and each chapter is translated once it is ready. When a chapter's translation does not exist yet, its button is gray.
Software
Every analysis in the book is shown in four programs: Excel, SPSS, JASP and R. R is for anyone who wants to write code (we will meet it in chapter 4). Choose the program you work with, and every "In software" section in the book will switch to it. You can also switch with the tabs inside each section. Try it here:
The mean in Excel: in an empty cell, type =AVERAGE(A2:A6) and press Enter.
The mean in SPSS: Analyze > Descriptive Statistics > Descriptives Variable(s) Move the variable into this box. OK
The mean in JASP: Descriptives > Descriptive Statistics Variables Drag the variable into this box. The mean appears right away, without clicking anything.
The mean in R: after you read in the file (chapter 4), you type one line and run it (Ctrl+Enter, or Cmd+Enter on a Mac).
mean(cs$sat_pre, na.rm = TRUE) # na.rm = TRUE: skip empty cellsR
[1] 5.064
There are analyses that one of the programs cannot do. When that happens, the book will simply say so, and suggest switching to a program that can.
Depth
Every topic has three levels of explanation. Try changing the depth in the top bar, and watch how the next passage changes:
In short: the mean is the sum of the values divided by how many there are.
Explanation: the mean is the value each person would have if we shared the total equally. If five students studied for 1, 5, 6, 8 and 10 hours, together they studied 30 hours. If each of them had studied the same number of hours, each would have studied 6. That is the mean.
The mean has a special property: it is the balance point of the data. If you place the values on a ruler, like weights, the ruler balances exactly at the mean. In chapter 11 we will see why this is true, and why it makes the mean so sensitive to outliers.
- Level 1 (In short): only the important sentences. Good for review before an exam.
- Level 2 (Explanation): the regular reading. This is the default.
- Level 3 (In depth): adds the "why": where a formula comes from, debates between researchers, and details that are usually skipped.
The worked examples, the exercises and the software sections appear at every level of depth.
Check yourself3 questions
1.In the top bar you chose the depth "In short". What will happen to the worked examples and the exercises?
2.In a certain chapter, the Arabic button in the top bar is gray. What does that mean?
3.Four teachers graded 10, 14, 12 and 18 tests in a week. Following the explanation at the "Explanation" level: if they had shared all the tests equally, how many would each teacher have graded?
Symbols you can click
Every statistical symbol in the book is colored purple. Click or and see what they mean, how other books write them, and what they are called in the output of SPSS and JASP.
The book writes symbols according to APA 7, the style guide used for writing papers in the social sciences and in education. That way, what you learn here will look the same in the papers you read and in the papers you write. In other books, and in other courses, you will sometimes meet a different way of writing the same idea. For example, the mean is also written as x̄. The appendix "Table of notation" has all the alternatives, and each symbol shows them in its own window too.
Check yourself3 questions
1.You clicked a purple symbol in the book. What will you not find in the window that opens?
2.A paper you read uses the symbol x̄. Which symbol in the book does it match?
3.Why does the book write symbols according to APA 7, and not the way a particular book or a particular lecturer writes them?
How each chapter is built
All the chapters are built in the same order, so you always know where you are:
- An opening question from the world of education, which the chapter answers.
- By the end of this chapter you will be able to: three or four goals.
- Explanation: first why, and only then how.
- A worked example, where every number is shown step by step.
- Your turn: a similar exercise, with the solution hidden behind a button.
- A lab: an interactive applet, with tasks.
- In software: the same analysis in Excel, SPSS, JASP and R, and how to read the output.
- Common mistakes.
- Check yourself: questions with instant feedback.
- Summary, formulas and new terms.
- Practice with the data: questions about a real file, with answers.
Check yourself3 questions
1.In a certain chapter you want to practice on a real file, and check your answers. Which part do you go to?
2.Which comes first in every chapter?
3.You have finished reading a chapter. You want to check quickly whether you understood, and then review the formulas. Which two parts of the chapter do you go to?
Labs
The book has more than 60 labs: small pages where you move a slider, change the data, and see what happens to the mean, to a graph or to a test. Each lab comes with tasks, and the answers are hidden. In the top corner of every lab there are two buttons. "⛶ Full screen" opens the lab on its own, on the whole screen. The second button switches between "⤡ Fit to screen", where the lab is shrunk a little so that all of it fits on one screen, and "⤢ Full size", its regular size. And on a computer there is also "🔍 Magnifier": when it is on, the part of the lab under the mouse is enlarged, so it is easy to read. In full screen the whole lab is enlarged, if there is room. Here is a taste from chapter 11: the mean as the balance point of the data.
Click "Add an extreme value (30)": this adds an outlier. Which one moves more: the mean or the median? (The median is the value in the middle, when you sort the values from smallest to largest. We will learn it in chapter 11.)
Check your answer
The mean moves a lot, and the median hardly moves at all. The mean uses every value and its size, so one outlier pulls it. The median only looks at the order. We will come back to this in chapter 11.
You can also find all the labs, arranged by chapter, on the All labs page, and at the top of the table of contents.
Check yourself3 questions
1.What do you do in one of the book's labs?
2.You are in chapter 5 and want to go back to a lab from chapter 11. Where will you find it?
3.The mean of 2, 4, 6 and 8 is 5. You replace 8 with 40. What is the mean now? And what does this say about the mean as a balance point?
How to learn from the book: stop and try
The best way to learn statistics is to try on your own, before you see the solution. That is why every chapter has exercises with hidden solutions. Read the question, try to solve it with pen and paper or in software, and only then click "Check your answer". Even an attempt that did not work teaches you more than reading a ready-made solution.
The same goes for software. The best guide is your own hands: after you do something once or twice yourself, it clicks. And don't try to memorize commands and menus. Try to understand the principle, and you will find the command on your own. And when something does not work, and it will happen, search the web. Almost every question about Excel, SPSS, JASP or R has already been asked, and the answer is one search away. The book's job is to show you where to look, not to replace your curiosity.
Two more things. First: statistics is built brick by brick. Every chapter rests on the ones before it, so it is worth not skipping, even when a chapter seems easy. Second: studying in pairs helps. When one of you gets stuck, the other helps. And when you explain something to someone else, you discover what is still not clear to you.
In my classes I sometimes open with a minute of mindfulness practice, and I do it on purpose before statistics, a subject many people feel stressed about. We learn best when we are calm. So before a hard chapter, try this: notice how you are sitting, and the feeling of your feet on the floor. Close your eyes if that feels comfortable, and notice your breathing without changing it. Relax your eyes, your jaw and your shoulders. Take one deep breath, and open your eyes. By the way, the study that runs through this book tests exactly this kind of program.
Notice, too, that new ideas sometimes appear "along the way", before the chapter that explains them in depth. For example, the z-score already appears in chapter 4, as an example of an Excel formula, and its own chapter is chapter 14. This is on purpose: a short first meeting prepares the ground for the full explanation.
Five students studied 1, 5, 6, 8 and 10 hours a week. What is the mean number of study hours?
Check your answer
A mean of 6 hours. And if you got it right without thinking twice, you are already ready for the next chapter.
Check yourself3 questions
1.You are working in software, and you don't remember which menu a certain analysis is in. What does the book recommend?
2.The z-score already appears in chapter 4, but its own chapter is chapter 14. Why?
3.The mean number of study hours for four students is 5. Three of them studied 3, 4 and 6 hours. How many hours did the fourth one study? Try before you check.
The study that runs through the book
Most of the examples in the book come from one study: a mindfulness and compassion program in elementary schools. 200 students in 4th and 5th grade, at three schools. In one school only the teachers took part in the training (the indirect program), in the second the teachers and the students took part together (the combined program), and the third was a control group. The students filled out questionnaires about life satisfaction, mindfulness, anxiety, attention problems, somatic complaints, their bond with the teacher and how they feel in class, three times: before the program, right after it, and six months after it. And one action was measured as well: whether the students volunteered to help other children.
The data are simulated, which means they were not collected from real students, but the design of the study is inspired by a real study carried out in schools in Israel (Tarrasch & Berger, 2022). That way you will get to know the file well, and in every chapter you will ask it a new question. In chapter 9 we will learn how the file is built, in chapter 23 we will test whether the programs lowered anxiety, and in chapter 28 we will compare the three groups.
Check yourself3 questions
1.How many times did the students in the study fill out the questionnaires?
2.At the school with "the indirect program", who took part in the training?
3.The data in the book are simulated. So what is the point of learning from them?
The data files
Every chapter that has practice with data comes with the files, along with a codebook that explains each column. For Excel and SPSS, an Excel or CSV file works. JASP opens both Excel files and CSV files. Watch out for one thing: from an Excel file it loads only the first sheet. So if the file has several sheets, make sure the data are on the first one. The book's data files are CSV files, and you can open them directly in all four programs.
Check yourself
In the next chapter: a math toolbox. The order of operations, fractions, percentages, rounding, and the summation sign Σ. Everything you need from school arithmetic, and no more.