Determine Whether The Given Set Of Functions Are Linearly Independent. If Not Express One Of The

Simply speaking, what is linear Independence?

Suppose you have orange juice, vodka, and a Screwdriver (cocktail). These are not independent because orange juice + vodka = screwdriver.On the other hand, suppose you have orange juice, vodka, and milk. These are independent because you can't add up any two to make the third, no matter the proportions.That is the basic idea of independence. It's when none of the elements of a set can be added up (or subtracted) in any proportion to make any of the other elements.For example,(1,0), (1,3), (-2,1)is dependent because1/7*(1,3) - 3/7*(-2,1) = (1,0)As another example(1,0,0), (1,3,0), (0,0,1)is independent because no combination of two of them can make up the third.If you just mess around for a while, you will probably discover some method or other to tell if a set is independent, for example something like Gaussian elimination. (The link might not be easy to understand if you don't know matrices, but I'm saying just to mess around with equations and you will probably discover the underlying methods in a form you do understand.)

What's the relationship between linear and logistic regression? What are the similarities and differences?

I will explain what is logistic regression and compare it with linear regression.Logistic regression falls under the category of supervised learning; it measures the relationship between the categorical dependent variable and one or more independent variables by estimating probabilities using a logistic/sigmoid function. In spite of the name ‘logistic regression,’ this is not used for regression problem where the task is to predict the real-valued output. It is a classification problem which is used to predict a binary outcome (1/0, -1/1, True/False) given a set of independent variables.Logistic regression is a bit similar to the linear regression or we can see it as a generalized linear model.In linear regression, we predict a real-valued output y based on a weighted sum of input variables.[math]y = c + x1*w1 + x2*w2 +x3*w3 +........+xn*wn[/math]The aim of linear regression is to estimate values for the model coefficients c, w1, w2, w3 ….wn and fit the training data with minimal squared error and predict the output y.Logistic regression does the same thing, but with one addition. The logistic regression model computes a weighted sum of the input variables similar to the linear regression, but it runs the result through a special non-linear function, the logistic function or sigmoid function to produce the output y. Here, the output is binary or in the form of 0/1 or -1/1.[math]y = logistic (c + x1*w1 + x2*w2 +x3*w3 +........+xn*wn)[/math][math]y = 1 / 1 + e [- (c + x1*w1 + x2*w2 +x3*w3 +........+xn*wn)][/math]The sigmoid/logistic function is given by the following equation.[math]y = 1 / 1+ e^-x[/math]`As you see in the graph, it is an S-shaped curve that gets closer to 1 as the value of input variable increases above 0 and gets closer to 0 as the input variable decreases below 0. The output of the sigmoid function is 0.5 when the input variable is 0.Thus, if the output is more than 0.5, we can classify the outcome as 1 (or positive) and if it is less than 0.5, we can classify it as 0 (or negative).References:Logistic regression - Wikipedia

Why does it matter if a set of vectors is linearly dependent or independent? I feel like my linear algebra text is beating the idea to death.

Collections of vectors are our first tools for exploring a finite-dimensional vector space. The two key properties of a collection [math]\mathscr{C}[/math] of vectors in a vector space V are:[math]\mathscr{C}[/math] being linearly independent, which means that each vector of V can be built as a linear combination of [math]\mathscr{C}[/math] in at most one way, and[math]\mathscr{C}[/math] spanning V, which means that each vector of V can be built as a linear combination of [math]\mathscr{C}[/math] in at least one way.If a collection [math]\mathscr{C}[/math] is both—if it’s linearly independent and spans V—it’s a basis for V, which is the key concept you’re working toward. A basis for V lets you write each vector of V in exactly one way as a linear combination of [math]\mathscr{C}[/math]. You can then think of each vector in V—whatever vector space that might be—as corresponding to a column vector of the coefficients it takes to build that vector from [math]\mathscr{C}[/math] (these are called [math]\mathscr{C}[/math]-coordinates). The correspondence is what’s called an isomorphism, and it means that V essentially acts like a vector space of column vectors, which puts all finite-dimensional vector spaces on level footing, because we understand how column vectors work pretty well (it also allows us to define the dimension of V)!So, in some way, if you’ve seen linear independence but not spanning yet, then linear independence is waiting for its mate…once you have both, you can get somewhere mathematically.

How would you show that a determinant can be zero if and only if its rows are linearly dependent?

Suppose the rows of an [math]n\times n[/math] matrix [math]A[/math] are given by [math]\vec{r_1}, \dots, \vec{r_n}[/math].First we want to show that if the rows are linearly dependent, then the determinant of [math]A[/math] is [math]0[/math]. Assume the rows are linearly dependent. Then there exists scalars, [math]c_1, \dots , c_n[/math], not all zero, such that [math]c_1\vec{r_1} + \cdots + c_n\vec{r_n} = \vec{0}[/math]. Take [math]c_k[/math] to be a nonzero weight, for [math]k \in [1,n][/math]. Then [math]\vec{r_k}[/math] is a linear combination of the other vectors. It follows that we can do a sequence of row operations to get a [math]0[/math] row in the [math]k[/math]th row (adding multiples of one row to another does not change the determinant). Thus, doing a cofactor expansion along the [math]k[/math]th row in the appropriate row equivalent matrix will leave us with a determinant of [math]0[/math].Next we want to show that if the determinant is zero, then the rows are linearly dependent. That’s equivalent to showing that if the rows are linearly independent, then the determinant is not zero (Proof by contrapositive). Suppose that the rows of [math]A[/math] are linearly independent and that [math]U[/math] is an echelon form of [math]A[/math]. Then the determinant of [math]A[/math] is some scalar multiple of the product of the diagonal entries in U. None of these entries can be zero, because otherwise one of the rows could be reduced to a row of zeros and by a similar argument from before, that would entail a relation of linear dependence on the rows. Thus, [math]\det(A) \neq 0[/math].A shorter argument would be to note that [math]A[/math] is invertable if and only if [math]\det(A) \neq 0[/math] if and only if [math]\det(A^T) \neq 0[/math] if and only if the columns of [math]A[/math] are linearly independent if and only if the columns of [math]A^T[/math] are linearly independent. Which works because the columns of [math]A^T[/math] are the rows of [math]A[/math].

Does the span of a set of vectors always go through the origin?

In answer to "Does the span of a set of vectors always go through the origin?"The span of a set of vectors is another set of vectors (always a superset, though not necessarily a proper superset).  Sets of vectors do not "go through the origin", though in general, they may, or may not, include the zero vector.So, the question is better asked "Does the span of a set of vectors always include the zero vector?".Starting with a set of vectors [math]\left\{ v_i\right\}[/math], the span of that set is defined to be every vector which can be expressed as a linear combination of elements of the set.  The 'zero' in any particular context is always a linear combination of any set of elements, and in this context, the zero vector is a linear combination of any set of vectors.So, yes.  The span of a set of vectors (even the empty set!) always includes the zero vector (and when the initial set is empty, the span contains exactly one element, the zero vector).