One of my paired sample t-tests came back at p=0.11. I don't know enough about stats to understand how far from being significant those results are? How would I quantify that?
As others have mentioned, p-values are arbitrary. What would be helpful is to examine why the p-value is 0.11. If you're using t-tests, then two factors generally are worth considering: the mean difference between pairs, and the variance associated with the sample. If you have a large mean difference but high variance, you could get a non-significant (and by significant, i mean statistically significant, not clinically/generally significant) value but it would likely suggest that either a) there's too much heterogeneity in the effect, at least based on this sample, to make reasonable conclusions about the differences observed in the pairs and/or b) you have a small sample size. If you have a very small mean difference, that would also lead to a non-significant value but in that case, it would likely reflect that there aren't any real differences instead. P-values are based on the number of data points used, hence the move for some scientific fields towards examination of effect sizes in addition to or in lieu of p-values.
Paired or Unpaired T-test?
It depends on your data type... The independent samples t-test compares the difference in the means from the two groups to a given value (usually 0). In other words, it tests whether the difference in the means is 0. The dependent-sample or paired t-test compares the difference in the means from the two variables measured on the same set of subjects to a given number (usually 0), while taking into account the fact that the scores are not independent. Now Unpaired and paired two-sample t-tests Unpaired : The unpaired, or "independent samples" t-test is used when two separate sets of independent and identically distributed samples are obtained, one from each of the two populations being compared. i.e., suppose we are evaluating the effect of a medical treatment, and we enroll 100 subjects into our study, then randomize 50 subjects to the treatment group and 50 subjects to the control group. In this case, we have two independent samples and would use the unpaired form of the t-test. The randomization is not essential here—if we contacted 100 people by phone and obtained each person's age and gender, and then used a two-sample t-test to see whether the mean ages differ by gender, this would also be an independent samples t-test, even though the data are observational. Paired: Dependent samples (or "paired") t-tests typically consist of a sample of matched pairs of similar units, or one group of units that has been tested twice (a "repeated measures" t-test). A typical example of the repeated measures t-test would be where subjects are tested prior to a treatment, say for high blood pressure, and the same subjects are tested again after treatment with a blood-pressure lowering medication. A dependent t-test based on a "matched-pairs sample" results from an unpaired sample that is subsequently used to form a paired sample, by using additional variables that were measured along with the variable of interest. The matching is carried out by identifying pairs of values consisting of one observation from each of the two samples, where the pair is similar in terms of other measured variables. This approach is often used in observational studies to reduce or eliminate the effects of confounding factors.
When should we use a paired T test?
When measurements on same group of objects are made twice, then in order to compare the difference paired T test is used... It is generally used to compare the effect of drug, method, strategical effect...
Is there a matched pairs statistical test for paired proportions?
The choice of test depends on the exact nature of your data; you may be looking for the McNemar test.
What is the difference between a paired and unpaired t-test?
Simple Explanation:T-tests are useful for comparing the means of two samples. There are two types: paired and unpaired.Paired means that both samples consist of the same test subjects. A paired t-test is equivalent to a one-sample t-test.Unpaired means that both samples consist of distinct test subjects. An unpaired t-test is equivalent to a two-sample t-test.For example, if you wanted to conduct an experiment to see how drinking an energy drink increases heart rate, you could do it two ways.The "paired" way would be to measure the heart rate of 10 people before they drink the energy drink and then measure the heart rate of the same 10 people after drinking the energy drink. These two samples consist of the same test subjects, so you would perform a paired t-test on the means of both samples.The "unpaired" way would be to measure the heart rate of 10 people before drinking an energy drink and then measure the heart rate of some other group of peoplewho have drank energy drinks. These two samples consist of different test subjects, so you would perform an unpaired t-test on the means of both samples.Source. What is a paired and unpaired t-test? What are the differences? | Socratic
Distinguish between situations requiring a two sample t-test and a paired sample t-test . What distributional assumptions are made in each case?
The Two Sample-t test for the Means, is conducted based on two equal size random samples drawn independently from two separate populations. The objective is to compare the populations for a particular Average (Mean) characteristic. This is done by setting up a Null Hypothesis of equality of the Means against an appropriate Alternative Hypothesis of Non--Equality. The assumptions are the samples are drawn from two independent Normal and "homoschedastic " populations. The term under " " means populations with same or equal variances. The Paired-t test is also a test for the Means, but based on a Single Random Sample but under two different conditions, or in a" Before" and " After" . This is usually favored when a test is needed to compare the effects of a " treatment" on a group of " units". Say, a fertilizer on a plant or a medicine in a trial. Observations are taken under two different conditions and then the averages compared. In this case also, the sample is taken form a Normal distributions with variance known. Appropriate Null hypothesis is set up with Alternatives to set up the correct test.One point to remember, in both cases, if the variances of the populations are unknown, then the unbiased estimates of the population variance is to be estimated from the samples. And that results in losing 1 degrees of freedom( d.f.) Initially, the sample means needed to be computed too for the test statistic, so 1 d.f. was lost already. Therefore, the final test statistic t will have n-2 d.f. If the sample size was of size n. May I also add, ( see comments) that as Peter F. has mentioned in his answer, the paired t test is also applied for a paired sample,(X, Y), every observation of X has a pair Y taken under the same conditions. I am adding this paragraph after he pointed it out. Although it is not part of the question, I like to add that most of the tests of significance assume Normality as basics. An enormous field of Non-Normality discusses various situations when the samples are drawn from distributions that are Not Normal. Then there are Non Parametric tests too.