Blog 7. What can influence research results, and how do researchers take this into account?

Researchers are trying to gain a better understanding of what happens in the body in ME/CFS. In previous blogs, we explained that they use different types of scientific research to do this. In this blog, we explain which factors can influence the results of ME/CFS research and what researchers do to make their studies as reliable as possible.

Confounding variables of verstorende variabelen

In science, the term confounding variables is used to describe factors that can interfere with research results. These can play an important role in biomedical research, including research into ME/CFS. After all, every person is different. People differ in factors such as age, sex, lifestyle and underlying medical conditions. These differences can affect how research results are interpreted.

Suppose researchers want to find out whether drinking coffee increases blood pressure. People who smoke tend to drink more coffee on average and often also have higher blood pressure. If researchers only look at coffee consumption and blood pressure, they may overlook another factor: smoking. This third variable is called a confounding variable. It may be associated with both the potential cause being investigated – in this example, drinking coffee – and the potential outcome, higher blood pressure. It would therefore be incorrect to simply conclude that drinking coffee causes the higher blood pressure.

When conducting research involving people, it is therefore important to carefully document participants’ characteristics. This allows researchers to identify potential confounding variables and take them into account.

Confounding variables can also play a role in biomedical research into ME/CFS. Suppose researchers are investigating differences in inflammatory markers in the blood of people with ME/CFS and healthy controls. Factors such as age, sex, medication use or other medical conditions may influence the results. It is therefore important to know these characteristics, include them in the study and, where appropriate, adjust for them. This allows researchers to better assess whether the differences they find are actually associated with ME/CFS.

How bias can arise in research

Another challenge is bias. Bias can cause research results to give an inaccurate picture of reality. It can arise at different stages of a study: when selecting participants, when collecting and measuring data, and when analysing and reporting results. Below, we discuss several types of bias that may also play a role in ME/CFS research.

• Selection bias: the group of participants is not representative of the entire population about which researchers wish to make claims. For example, if a study is conducted only among healthy young men, the results cannot simply be applied to older women in poorer health.
• Attrition bias: occurs when certain participants are more likely to drop out of a study than others. For example, severely ill participants may be more likely to drop out than participants with milder symptoms, which could make the study results appear more positive than they really are. This can also occur in ME/CFS research. Taking part in research can be demanding, which means that not everyone is able to complete an entire study. As a result, data from these participants may be missing.
• Information bias : arises from errors in collecting, measuring or reporting information. Participants may, for example, not remember exactly when they were exposed to certain risk factors. Researchers may also unintentionally influence a measurement or the interpretation of data.
• Publication bias: positive or striking research findings are more likely to be published than negative findings. This can create a distorted picture of what the research as a whole shows.

Bias cannot always be completely prevented. Researchers therefore try to minimise the risk of bias as much as possible. This begins when designing a study, for example by carefully considering how participants are selected and how data are collected and measured. Researchers can also take potential sources of bias into account during the analysis. Later in this blog, we discuss several methods researchers use to make research results as reliable as possible.

How can measurement errors influence research results?

Measurements themselves can also influence research results. Measuring equipment and laboratory tests, for example, may have small inaccuracies. Different devices or different research sites may also produce slightly different values for the same measurement. If a device consistently produces a value that is slightly too high or too low, this deviation will occur in all measurements. This is called a systematic error.

In addition, no measurement is completely precise. There is always a certain degree of measurement uncertainty. For example, a thermometer that only measures in whole degrees Celsius cannot reliably detect differences of half a degree. Biological variation also plays a role in research involving people. People differ from one another, but values can also change within the same person. Blood pressure, for example, can vary throughout the day. The time at which researchers take a measurement can therefore influence the results. Larger study groups are often needed to reduce the influence of this type of variation. In a small group, chance differences between participants or fluctuations in measurements can have a substantial effect on the results, whereas larger groups provide a more reliable picture of what happens on average.

Data analysis and reporting

Challenges can also arise when analysing and reporting data. In biomedical research, researchers sometimes investigate many possible associations at the same time. This is the case, for example, in broad studies of genetics, metabolism or proteins. These are also known as omics studies and are also used in ME/CFS research.

The more tests researchers perform, the greater the chance that they will find an apparent effect purely by chance, even though no real effect exists. This is called a false-positive result. The opposite can also occur: a study may find no effect even though an effect actually exists. This is called a false-negative result.

Choices made during the analysis can also influence the outcome. For example, a researcher may try different analyses until a result emerges that appears significant. This can happen consciously or unconsciously and is known as p-hacking. It can make it appear that an association exists when in fact it does not.

With HARKing, researchers formulate a hypothesis only after seeing the results and then write the article as though they had developed that hypothesis before conducting the study. This can make an association found by chance appear more important than it actually is.

How do researchers ensure that research is reliable?

Many things can happen during a study that influence its outcome. Sometimes this happens deliberately, but often without researchers being aware of it. So how do researchers minimise these influences and make their results as reliable as possible? They use a range of methods, including:

• Randomisation: participants are randomly assigned to study groups. This helps distribute confounding variables as evenly as possible across the groups.
• A larger sample size: reduces the influence of chance differences and generally makes results more precise. However, a larger sample does not automatically prevent bias or measurement errors.
• Control groups: help researchers assess whether an observed effect is actually caused by, for example, a treatment or by other factors.
• Blinding: helps prevent the expectations of participants or researchers from influencing the results. In a single-blind study, participants do not know which treatment they receive. In a double-blind study, the researchers do not know either.
• Objective measurements: help limit the influence of human expectations and interpretations.
• Statistiscal technics: help researchers account for confounding factors, variation and uncertainty in the data.
• Replication and verification: when different independent research groups find similar results, confidence in the reliability of the findings increases.

Researchers often use several of these methods within a single study. Randomised Controlled Trials (RCTs), for example, combine randomisation, control groups and blinding. RCTs are widely used to investigate whether a treatment or other intervention actually has an effect.

Onderzoek is meer dan alleen meten

Good research is therefore about more than simply what is measured. Researchers must also consider who takes part, how measurements are carried out, which other factors may have an influence, and how the results are analysed. By taking confounding variables, bias, measurement errors and other forms of uncertainty into account, researchers can better support their conclusions.
Even when a study has been conducted carefully, however, review and verification by other scientists remain important. In the next blog, you can read more about how this works.