Experimental Design, Sample Collection and Preparation

.title[
# Experimental Design, Sample Collection and Preparation
]
.author[
### <a href="https://shibalytics.com/">Max Qiu, PhD</a> </br> Bioinformatician/Computational Biologist </br> <a href="mailto:maxqiu@unl.edu" class="email">maxqiu@unl.edu</a>
]
.institute[
### <a href="https://biotech.unl.edu/bioinformatics">Bioinformatics Research Core Facility, Center for Biotechnology</a> </br> <a href="https://ncibc.unl.edu/">Data Life Science Core, NCIBC</a>
]
.date[
### 02-08-2023
]

---

background-image: url(data:image/png;base64,#https://media.springernature.com/full/springer-static/image/art%3A10.1186%2Fs13024-018-0304-2/MediaObjects/13024_2018_304_Fig1_HTML.png?as=webp)
background-size: 75%

# MS Omics Workflow

.footnote[
[Shao, Y., Le, W. Mol Neurodegeneration 14, 3 (2019)](https://doi.org/10.1186/s13024-018-0304-2)
]

???

Previously:

"Proteomics is the attempt to understand which proteins are doing what, when, with whom, and why." It collects information about **location, abundance/turnover, and PTMs** of proteins, and additional inferences about protein-protein interaction and activity. It **reflects underlying transcriptome**, but is also **regulated by geneome-unrelated factors**. Two main approach: bottom-up and top-down proteomics.

Metabolomics is a non-biased experimental approach that attempts to measure all of the metabolites (small molecures) in a biological sample. It is a **high-throughput** approach, and it provides the **closest link to the phenotype** of an organism. Two main approach: targeted and untargeted metabolomics.

---

# Experimental Design

Experimental design is the design of a data collection study where **all sources of variation** are assessed in the design process and contribute to the final project design.

* Goal: Observed changes due to factor of interest **only** (study question)
  + measure the variation in one source while controlling all other constant

???

Experimental design or the design of an experiment is a critical process in any scientific study.

A simple definition of experimental design is ... **An important thing to consider is that the different sources of variation MAY or MAY NOT be under the control of the researcher**. Ideally, we would like to measure the variation in one source while controlling all other sources of variation.

???
For example, when identifying the metabolic changes resulting from a genetic change in yeast, all sources of variation will be controlled such as the culture media and temperature - the only difference in the experiment will be the genotype.

---

# Experimental Design

Experimental design is the design of a data collection study where **all sources of variation** are assessed in the design process and contribute to the final project design.

* Goal: Observed changes due to factor of interest **only** (study question)
  + measure the variation in one source while controlling all other constant

???

By controlling all sources of variation except the one you are testing - in this example, the genetic difference in yeast, we are confident that our observations are due to the parameter we are testing, and not another source of variation or combination of sources of variation.

---

# Experimental Design

* Depends on type of study
  + **Lab-controlled study**
  + Population study: genotype, environment and lifestyle

???

**Whether we can control other sources of variation is dependent on the type of study**. In the lab, we can normally control the genotypes and environment when studying microbes, plants, or mammalian cell lines, and so experimental design is relatively simple.

---

# Experimental Design

* Depends on type of study
  + Lab-controlled study
  + **Population study: genotype, environment and lifestyle**

???

When studying samples from the human population there are many sources of variation - related to the genotype, environment, and lifestyle.

---

# Experimental Design

* Population study
  + Many sources of variation: hard to control
  + Instead, ensure **variability is equivalent**

???

In many studies, it is very difficult to control these sources of variation and so instead, **we need to ensure that the variability is equivalent between different groups of subjects**. For example, we want to ensure the age range and the ratio of males to females is equivalent in the sample groups - because we know these will influence the phenotype and therefore the measured metabolome. Other factors such as food intake, medications, and many more, should also be considered in human studies.

---

# Experimental Design

* Population study
  + Many sources of variation: hard to control
  + Instead, ensure **variability is equivalent**

???

**Where these sources of variation are not matched, then the study will be biased and the metabolic changes observed are a result of two or more sources of variation.** So if you are studying metabolic differences between a group of subjects who have a disease and a group of subjects who do not have a disease, and the control group are aged 30-45 years and the disease group is aged 50-60 years. You have **two sources of variation**, age and disease. So the observed changes in the metabolome are due to both variations. If the age range of both groups is similar, then the influence of age is removed and the study now only have one source of variance, the disease. Then you can be confident that your observed changes in the metabolome are due to the disease only.

---

# Experimental Design

* Sample numbers
  + Define at the start of the study to ensure **statistical robustness** (**statistical power**)
  + **Pilot studies** are particularly helpful with study design for this reason.
  + Power analysis to decide `\(n\)` if you have pilot data

???

Apart from controlling sources of variances, sample numbers are an important consideration in experimental design, and should be **defined at the start of the study** to ensure an **appropriate number of biological replicates** in each biological groups to **confidently** answer the question in a **statistically robust manner**.

Here **statistically robust manner** refer the issue of **statistical power**, which is an very important issue in experimental design phase that researchers most frequently neglect. This is understandable, because we are at the beginning of an experiment, which is why **pilot or preliminary studies, or even thorough literature review of similar studies** are particularly useful when it comes to deciding sample numbers.

To fully understand statistical power, you need to trace way back to STAT 801 and 802, where the **fundamentals of inferential statistics** was discussed, including normal distribution, sample and population, sampling distribution and variability, central limit theorem, hypothesis testing and test statistics, confidence interval and confidence level.

---

# Statistical Power

**Power is the probability of detecting an effect, given that the effect is really there.**

* Example: if a study comparing the two groups (control vs disease) has a power of 0.8, and assuming we can conduct the study many times, then 80% of the time, we would get a statistically significant difference between the two groups, while 20% of the time we run this experiment, we will not obtain a statistically significant effect, even though there really is an effect in reality.

* In practice, we are most interested in knowing the **sample size required in order to obtain sufficient power**.

---

# Statistical Power

* Type 1 error: rejecting `\(H_{0}\)` when you shouldn't have, and the probability of doing so is `\(\alpha\)` (significance level).

* Type 2 error: failing to reject `\(H_{0}\)` when you should have, and the probability of doing so is `\(\beta\)`.

* **Power** of a test is the probability of correctly rejecting `\(H_{0}\)` and the probability of doing so is `\(1-\beta\)`

### Goal: keep both `\(\alpha\)` and `\(\beta\)` low

???
Here's a two by two table that basically tells us what the true state of the hypotheses are, and remember we usually don't know whether the null hypothesis or the alternative hypothesis is true, but we make a decision regardless based on the evidence that we collect or on the statistical significance of that evidence.

If null hypothesis is true and we fail to reject, then we did the right thing. Similarly, if alternative is true, and we successfully rejected the null, once again we did the right thing. But how about the other two cells?

**A type 1 error is rejecting the null hypothesis when the null hypothesis is actually true**. So in other words, rejecting the null hypothesis when you should not have. The probability of doing so is alpha, which is the p-value. This is **something we set**, usually we set it at 0.05, meaning that we are **accepting 5% chance that we reject the null when we shouldn't have**.

**A type 2 error is failing to reject the null hypothesis when the alternative is true**. In other words, it's failing to reject the null hypothesis when you should have reject. The probability of doing so is beta, this is **not something we get to set** ahead of time, and its calculation is directly related to the statistical power.

(Power is the probability of detecting an effect, given that the effect is really there.) **The power of a test is the probability of correctly rejecting the null and the probability of doing so is 1-beta.**

Our goal, in general, in hypothesis testing, is to **keep both alpha and beta low at the same time**. But we know that as we push one down, the other is going to shoot up. So we usually want to find a delicate balance between these two probabilities.

---

# Statistical Power

### How to increase power?

If `\(H_{a}\)` is true (there is really an effect), what is the chance that we detect that effect? 
.pull-left[

* **Effect Size ( `\(\delta\)` )**: True difference between null and alternative

* If the true population mean is very close to the null value (**effect size is small**), it will be very difficult to detect a difference (and reject `\(H_{0}\)`) (**green line**)

* If the true population mean is very different from the null value (**effect size is large**), it will be easier to detect a difference. (**pink and grey lines**)

* Power depends on the **effect size ( `\(\delta\)` )**, difference between point estimate and null value.

]

<img src="data:image/png;base64,#./img/power.png" width="100%" style="display: block; margin: auto;" />
]

.footnote[
[Statistical power and power analysis in python](https://machinelearningmastery.com/statistical-power-and-power-analysis-in-python/)
]

???

If the alternative hypothesis is actually true (there is really an effect), what is the chance that we detect the effect? The answer to this is **not obvious**, because it concerns the **effect size**, true difference between null and alternative.

If the effect size is small (**green line**), it will be very difficult to detect a difference and to reject the null hypothesis. If the effect size is large (**gray and pink line**), it will be much easier to detect a difference.

Clearly then, power depends on our effect size.

This is why we said before, that pilot or preliminary studies are useful when deciding sample size, because we can get an **estimated effect size** using your pilot data. With that information, we can then decide the sample size so that we can achieve an acceptable statistical power.

---

# Experimental Design

???
Going back to where we left off in experimental design.

In the lab environment where **variability is highly controlled**, fewer replicates are required, 6-10 biological replicates should be the minimum per biological group.

However, in studies where **variability is not controlled well**, when studying the human population, then hundreds or thousands of biological replicates are required to provide statistically significant and robust results.

---

# Experimental workflow is an upstream-to-downstream process.

???

It is important to remember that experimental workflow is an upsteam-to-dowstream process.

To be specific, steps that comes first has more impact to the final results than steps coming later.

More importantly, don’t expect later steps to compensate what was done (or not done) before.

By the time an experiment reaches data acquisition phase, more than 50% of the outcomes has already been decided by choices made in the prior stages. By the time batches of data reaches bioinformatics, major results of the study is fixed. Complicated data transformation and normalization cannot increase statistical power for high dimensional data analysis when an experiment contains 5 phenotypes with only 3 samples each. Machine learning cannot remedy the lack of quality control during weeks long LC-MS/MS data acquisition, or less-than-robust SOPs that will probably not stand the test of reproducibility.

Think hard early on, and consult with bioinformatician and statistician early on.

## What comes first has more impact than what comes later;

## Don't expect later steps to compensate what was done wrong before.

---

# Sample Collection and Preparation

* Consider sample type

* Impact of sample type on sample collection

* Sample storage

* Sample extraction

???

Moving on to sample collection and preparation. As we discussed before, metabolome is **dynamic** in nature. It can change in seconds and minutes. Even proteome changes slower in repose to perturbation and treatment, proteins in a sample will still degrade fast due to the many enzymes and protease.

If samples are not collected, processed, stored and extracted properly, then the **metabolites are very likely to have changed and proteins of interest will likely be degraded** and results will definitely not be accurate.

Within the context of sample preparation, we will discuss sample type, and how it affect sample collection, how sample should be stored and extracted.

---

# Sample Collection and Preparation

* **Consider sample type**
  + Urine
        - Time-averaged
        
        - Small proteins
        - Extreme dynamic range

+ Blood (Plasma or Serum)
        - Instantaneous snapshot
        - Homeostatic
        
        - Large protein/complexes

+ Other sample types includes CSF, saliva, sweat, feces, breath ...
]

]

???

The choice of sample is dependent on many factors, including the **biological question being asked, and the availability of appropriate samples**.

If we think about the question in **clinical studies**, if we are searching for a **biomarker of a disease to apply in clinical practice**, then we would study a **biofluid** like blood or urine, as this sample is easy to collect for clinical practice.

However, if we were interested in the **molecular mechanism** of a specific disease, then we would
prefer to investigate a sample where the **mechanism is in operation**, cells or tissue, rather than a biofluid.

---

# Sample Collection and Preparation

* **Impact of sample type on sample collection**

+ Metabolically active samples: cells or tissues
        - **Metabolic Quenching** to stop metabolic turnover and preserve metabolic profile at the point of sampling
  + Metabolically not active samples: microbial culture
        - Quenching not required, sample provide a cumulative picture of biology over time

???

Depending what sample you are analyzing, sample collection procedure varies too.

If samples that are **metabolically active** - cells and tissues - metabolism should be rapidly inhibited through **metabolic quenching** to stop enzymatic activity from progressing.

If samples that are **not metabolically active** - cell culture which do not contain enzymes - metabolic quenching is not required. These samples provide a **cumulative or averaging** picture of biological process over time rather than a snapshot. |

--
* **Sample Storage**
  + Prevent enzymatic activity and degradation of metabolites/proteins

???

All samples for Omics analysis should be stored in chilled conditions at -80C or if this is not possible, -20C to ensure that **enzyme activity does not return** and to **minimize degradation of unstable metabolites/proteins/peptides**. |

* **Sample Extraction**
  + Start at a chilled condition, -80&deg;C or -20&deg;C
  + Standardize the workflow: **Quench, Extract, and Store**
  + Extraction Solvent (focusing on metabolomics)
        - Water-soluble: water and methanol
        - Lipid-soluble: chloroform, etc

???

In the extraction, **samples from different groups should be treated in the same way**, started from a chilled condition. Experimental procedures should be **standardized and optimized** to quench, extract and store in the same way.

Extraction protocol is very diffenrent between metabolomics and proteomics. Focusing on metabolomics first. Depending on what type of metabolites you are interested in, **choice of solvent is different**. For metabolites, chloroform is applied to extract lipid-soluble metabolites, while methanol and water are applied to extract water-soluble metabolites.

---

# Sample Collection and Preparation

* **Sample Extraction (focusing on metabolomics)**

???

Extraction methods is also different depending on the sample type.

For urine, if LC-MS, simply dilute and analyze. If GC-MS, **urease treatment** can be applied to degrade urea, which is in large excess compared to other metabolites.

Serum or plasma requires the **removal of proteins** prior to MS so as to stop blockages in the chromotography system. This is typically performed by the addition of solvents - methanol or acetonitrile, which precipitates the proteins.

Tissue and cell samples requires addition **mechanical approaches** to assist with the liberation of metabolites from the cells. **Homogenizers and tissue lysers** may be needed for tissue samples to homogenize and break open cell, Multiple **freeze-thaw cycles** may also be needed to sufficiently permeabilize the cell walls.

---

# Sample Collection and Preparation

* **Sample Extraction (focusing on proteomics)**

[Protein Sample Preparation for Mass Spectrometry](https://www.thermofisher.com/us/en/home/life-science/protein-biology/protein-biology-learning-center/protein-biology-resource-library/pierce-protein-methods/sample-preparation-mass-spectrometry.html)
]

???
Extraction methods is also different depending on the sample type.

For the extraction of proteins, the use **protease inhibitor** is usually one of the first steps, to stop proteins being degraded and preserve protein profile at the point of extraction. Following steps include **cell lysis, subcellular fractionation, depletion of high-abundance proteins, enrichment of target proteins, dialysis and desalting**.

(Shotgun proteomics) **In-solution digestion** entails irreversibly breaking disulfide bonds via reduction and alkylation followed by protein digestion into peptide fragments.

An alternative approach is to **first resolve proteins by 1D or 2D electrophoresis** (1DE or 2DE, respectively) and then collect gel slices that contain the desired band(s). The proteins in these gel plugs are then reduced, alkylated and digested in situ. After peptides are extracted from the gel matrix, the **peptides are enriched and salts and detergents removed** and prepared for MS analysis.

---

# Sample Collection and Preparation

* **Sampling bias**
  + "In statistics, sampling bias is a bias in which a sample is collected in such a way that some members of the intended population have a lower or higher sampling probability than others."

* Technical replicates vs biological replicates

* Pseudo-replication
]

]

???

Wikipedia says "In statistics, sampling bias is a bias in which a sample is collected in such a way that some members of the intended population have a lower or higher sampling probability than others."

---

# Sample Collection and Preparation

* Sampling bias

* **Technical replicates vs biological replicates**
  + Technical replicates are repeated measurements of the same sample that show independent measures of the noise associated with the equipment or the protocols. 
  + Biological replicates are measurements of biologically distinct samples that show biological variation.

* Pseudo-replication
]

]

???

Technical replicates are repeated measurements of the same sample that show independent measures of the noise associated with the equipment or the protocols.

Biological replicates are measurements of biologically distinct samples that show biological variation.

Passing a technical replicate as a biological replicate is **pseudo-replication**.

---

# Sample Collection and Preparation

* Sampling bias

* Technical replicates vs biological replicates

* **Pseudo-replication**
  + when subjects are not independent of each other, but you count them as they were
]

???

**Pseudo replication: when subjects are not independent of each other, but you count them as they were.** It gave the impression that **there are more data/replication than there really is**. You can think of this as having **false confidence**. It will cause an **inflated probability of a type 1 error** (falsely rejecting a true null hypothesis).

Examples:
1. Suppose a blood-pressure lowering drug is administered to a patient, then the patient's blood pressure is measured twice. **This is a repeated measure, a technical replicate, not a biological replicate**. It can give information about the **uncertainty in the measurement process**, but not about the **variability in the effect of the drug**.

2. A researcher is studying the effect on plant growth of different concentrations of CO2 in the air.  He needs to grow the plants in a growth chamber so that the  CO2 concentration can be controlled. He has access to only two growth chambers, but each one will hold five plants. The five plants within each chamber are not independent replicates but are pseudo-replicates. **The growth chambers are the experimental units; the treatments are applied to the growth chambers, not to the plant independently**.

---

# Quality Assurance and Quality Control

**Quality assurance and quality control provides a mechanism to ensure that a scientific process meets the predefined criteria.**

* Quality Assurance
  + Before Data acquisition to ensure processes are operating correctly
  + Training of the staff, maintenance and calibration of the instruments

* Quality Control
  + During or after data acquisition to assess the quality of the data acquired
  + Internal standards (IS) and QC samples

]

]

---

# Next: Data Acquisition using Mass Spectrometry

Slides created via the R package [**xaringan**](https://github.com/yihui/xaringan). Sections of this lecture were modified from "Understanding Metabolism in the 21st Century" and "Metabolomics Data Processing and Data Analysis" workshop materials given by [**University of Birmingham**](https://www.futurelearn.com/partners/university-of-birmingham)