Why doesn't the definition of dependence require that one can expresses each vector in terms of the others?

Question

I was reviewing my foundations on linear algebra and realized that I am confused about independence and dependence. I understand that by definition independence means:

A set of vectors $\{x_1,\ldots,x_k\}$ is independent if the only linear combination that gives the zero vector is the zero vector itself. i.e. if $[x_1, \ldots, x_k]c = Xc = 0$ iff $c=0$

I understand what the definition says but it sort of goes against my intuition of what the definition of dependence should be (and hence its negation independence). In my head intuitively dependence means that the a set of vectors depends on each other. In other word one should always be able to express one vector as a linear combination of the others. Something like:

$$ \forall x_i \in \{x_1,\ldots,x_k\}, \exists c \neq 0 : \sum_{j \neq i} c_j x_j = x_i$$

however with my definition above (which is wrong and is not what the standard definition is, I know but I am trying to come to terms why its wrong) implies that a set of independent vectors with the zero vector tacked on is not dependent (i.e. independent) which is the opposite of what is should be. i.e. tacking the zero vector and the set remains independent (this should be wrong cuz [0,...,0,1] is not the zero vector and only the zero vector should give 0).

Consider for a simple example $ \{ x_1,x_2,0 \}$ where $x_1,x_1$ only give zero with the zero vector (standard definition of independence). With my definition of things its obvious that these vectors are independent. In reality they should be dependent because [0,0,1] is now in the nullspace but things are only independent if only the zero vector is in the nullspace. With my definition the vectors are independent because there is no way to express any of them in terms of each other. For example:

1. $a x_1 + b x_2 = 0$
2. $c x_1 + d 0 = x_2$
3. $e x_2 + f 0 = x_1$

non of the above can be made true with non zero (non-trivial) linear combinations. Thus, the vectors are not dependent so they are independent. I know its sort of an "edge case" condition for the definition but it sort of flipped my world to find out that I've been thinking about such a fundamental concept like independence and dependence wrongly in linear algebra and I'm trying to come to terms with it.

Why is my intuition incorrect? Why was the standard definition of independence as $Xc = 0 \iff c=0$ the accepted definition of independence? Whats wrong with my definition? Are they essentially the same definition except for this weird edge case?

---

last footnote is about what the word dependence means with respect to the number and vector zero. I think what my last confusion boils down to is why $0x = \mathbf{0}$ is considered as $\mathbf{0}$ depending on $x$. I guess in my head saying that we don't need any of $x$ to express $\mathbf{0}$ seems to mean that $\mathbf{0}$ doesn't need $x$ (or any other vector). But the convention according to everything pointed out by everyone in these set of answers points out to the opposite. I don't understand why. Is it that just having an equation linking terms means dependence even if we specify with a zero that we don't actually need the term?

Answer 1

Your intuition for linear (in)dependence is very close. Based on your intuition, the definition you're looking for is:

$\{v_1, ..., v_k\}$ is linearly dependent if there exists an index $i$ and scalars $c_1, ..., c_k$ (excluding $c_i$) such that $v_i = \sum_{j \ne i} c_j v_j.$

You can prove that this is equivalent to the standard definition.

Notice how this differs from your proposed definition:

(1) It says there exists a $v_i$, not for all $v_i$.

(2) There is no zero restriction on the $c_i$.

(1) is important because all it takes is a single redundancy to get linear dependence. Not all vectors have to expressible in terms of the others. To see why this is the case, just think about the case where a set $\{v_1, \ldots, v_k\}$ is already dependent and then I suddenly add a $v_{k+1}$ which cannot be expressed as a linear combination of $v_1, \ldots, v_k$. Adding a vector to a dependent set shouldn't turn it into an independent set.

As for (2), the standard definition needs to say that $c$'s can't be all 0 because you don't want $\sum 0 v_i = 0$ to imply dependence. But with the above definition, you've already singled out a vector to have a coefficient of 1 (which is not 0) so you don't need any condition on the c's anymore.

Answer 2

Let me address your last question (and hopefully it will help with clarifying some of your misconceptions):

Are they essentially the same definition except for this weird edge case?

No, not only in that case. Consider e.g. the following set of three vectors in $\mathbb{R}^2$:

$$\mathbf{v}_1=\begin{bmatrix}1\\0\end{bmatrix}, \quad \mathbf{v}_2=\begin{bmatrix}2\\0\end{bmatrix}, \quad \mathbf{v}_3=\begin{bmatrix}0\\1\end{bmatrix}.$$

It's easy to see that this set is linearly dependent according to the standard definition because $$2\mathbf{v}_1+(-1)\mathbf{v}_2+0\mathbf{v}_3=\mathbf{0}.$$ However it doesn't satisfy your definition. Although vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ can be expressed as (nontrivial) linear combinations of the other ones, viz. $\mathbf{v}_1=0.5\mathbf{v}_2+0\mathbf{v}_3$ and $\mathbf{v}_2=2\mathbf{v}_1+0\mathbf{v}_3$, we can't do the same with the last vector because the equation $$\mathbf{v}_3=c_1\mathbf{v}_1+c_2\mathbf{v}_2$$ clearly has no solutions.

Let me try to describe informally what I think is going on here. The standard definition of linear dependency basically says that there's some dependency somewhere, but not necessarily everywhere, as you seem to believe.

As @AOrtiz already said, one way to think of dependency is that it means redundancy in the given system of vectors. Look at it this way. Given a set of vectors, we may want to construct its span, i.e. the set of all linear combinations of those vectors. If the original set is linearly dependent, then it's redundant in the sense that you can remove some (but not arbitrary!) vectors and still have the same span. The standard definition of linear dependence helps us detect if that's the case.

Answer 3

I find that many of my students think the same way. Instead of thinking about null linear combinations, they usually prefer to think in terms of vectors as linear combinations of other vectors. And honestly, I probably do too. The definition of linear independence that is most intuitively geometric to me, is that no vector in the list can be expressed as linear combination of the others. This is equivalent to the other definitions of linear independence.

The negation of this is that some vector (not all vectors) in the list can be written as a linear combination of others. That is linear dependence. It has nothing to do with non-zero linear combinations (otherwise, as you pointed out, adding $0$ to the list will preserve linear independence). The zero vector is always a linear combination of the other vectors, adds nothing to the span, and therefore nothing to the dimension.

There are other cases, aside from $0$, where not every vector in a linearly dependent list can be expressed a a linear combination of others. For example,

$$((1, 0), (2, 0), (0, 1))$$

Some vectors (i.e. $(1, 0)$ and $(2, 0)$) can be expressed as linear combinations of the others, but not all. There is still dependency in the list.

Hope that helps.

Answer 4

I would prefer you state your definition of linear independence thusly:

Definition: The subset $\{v_1,\dots,v_n\}\subset V$ is linearly independent if whenever $a_1,\dots,a_n\in F$ and $$ a_1v_1+\dots+a_nv_n = 0, $$ then $a_1 = \dots = a_n = 0$.

Let's see how your intuition breaks down:

Definition: A set $A=\{v_1,\dots,v_n\}\subset V$ is linearly dependent if for each $v_i\in A$ there is a nontrivial linear combination $$ a_1v_1+\dots+\widehat{a_iv_i}+\dots+a_nv_n = v_i, $$ where the notation $a_1v_1+\dots+\widehat{a_iv_i}+\dots+a_nv_n$ means that $a_iv_i$ is excluded from the sum.

This says that every vector in the set $A$ can be expressed as a nontrivial linear combination of the other vectors. Well, what if we consider the set $A=\{e_1, 0\}$, where $e_1 = \begin{bmatrix} 1 \\ 0\end{bmatrix}$ is a column vector in $\Bbb R^2$, and $0$ is the zero vector. Then, according to our definition, this set $A$ is not dependent, since we can't express $0$ as a nontrivial linear combination of $e_1$. However, we expect this to be dependent because of course $0$ does depend on $e_1$, as in $0 = 0e_1$. [As I said in my comment to the OP below, I don't like how I originally phrased this—I would rather explain the intuition for why $\{e_1,0\}$ is dependent solely in terms of the redundancy that $0$ brings to this set.]

A better intuition for linear independence is that a set is linearly independent if we are specifying a minimal amount of information for the space it spans. That is, we can always consider the span of a set of vectors $A\subset V$. If we specify the minimal amount of information to achieve the span of $A$, then there are no redundancies: the vectors are independent. So dependent sets should be ones where we can find redundant information leftover.

To be concrete about the idea of how dependence $\leftrightarrow$ redundancies, consider the set $\{e_1,0\}$ again; this time consider its span too, i.e., $\{a_1e_1 + a_20:a_1,a_2\in F\}$. The $0$ vector is redundant because $\operatorname{span}(\{e_1,0\}) = \operatorname{span}(\{e_1\})$. Thus the $0$ vector is redundant, and the set is dependent.

On the other hand, if a set is independent, like $A=\{e_1,e_2\}\subset\Bbb R^2$, then we should not be able to remove even one vector from the set $A$ without changing $\operatorname{span}(A)$. This bears itself out here of course—reinforcing the intuition that independence $\leftrightarrow$ specifying the minimal amount of information.

Answer 5

To say that one of the vectors is a linear combination of the others singles out a vector to play a different role from the others. And it's possible that there are some among them that are not linear combinations of the others, but also some that are.

The point of the conventional definition is to make a statement in which none of the vectors plays a role different from the roles of the others, at least in the statement of the definition.

Answer 6

Why is my intuition incorrect?

I posit your intuition is incorrect because you learned from a biased source.

You probably learned about the idea of independence from talk of (in)dependent variables introductory calculus. However, dependence there is not spoken in a general sense, but is instead oriented towards a very specific application.

Specifically, actual problems are often most naturally expressed in terms of related variables, but introductory calculus tends to be presented in a very function-oriented manner. Thus, one is taught to re-express such problems in terms of functions, the typical method being to single out one or more variables (the 'independent variable(s)') to be used as function inputs, and interpreting the remaining variables as function outputs.

The general definition of independence in this setting is actually of the following form: a collection of variables are independent if and only if the only function $f$ satisfying $f(x_1, x_2, \ldots, x_n) = 0$ is the zero function.

You can also talk about more nuanced cases of independence, such as continuously independent ($f$ is restricted to continuous functions), differentiably independent ($f$ is restricted to differentiable functions), analytically independent ($f$ is restricted to analytic functions)... and, of course, the case at hand: linearly independent ($f$ is restricted to linear functions).

---

Incidentally, the fact that independence can be expressed in terms of comparisons to zero is a sort of weird quirk that is often applied to simplify definitions; the point may seem more intuitive when expressed in the following equivalent form:

A collection $\{ x_1, \ldots, x_n \}$ of vectors is linearly independent if and only if, whenever $f$ and $g$ are linear functions satisfying $f(x_1, \ldots, x_n) = g(x_1, \ldots, x_n)$, then $f = g$.

Answer 7

I think if you look at "dependent" as the negation of "independent" instead of the other way around, it'll make sense to you.

Independent is the lack of any dependence. So if there is even the tiniest dependence (between only a subset of the vectors), the whole set of vectors is dependent.

Your proposed definition requires a dependence relation between all of the vectors, which is just a "higher level of dependence" (so to speak) than is required to negate "independent".

Answer 8

Let me try to give you some intuition on the term independence, since you are lacking one so far. The usual definition is indeed

\begin{align} \{ v_1 , \dots, v_n \} \text{ are linearly independent} :\Longleftrightarrow&~ \textstyle \Big( \forall \alpha_j~~ \sum_{j = 1}^n \alpha_j v_j = 0 ~\Longrightarrow~ \forall j ~~\alpha_j = 0 \Big) ~~~~(\ast) \\ \Longleftrightarrow&~ \textstyle \Big( \forall \alpha_j~~ \sum_{j = 1}^n \alpha_j v_j = 0 ~\Longleftrightarrow~ \forall j ~~\alpha_j = 0 \Big) \end{align}

The last line following from the fact that $\forall j ~~\alpha_j = 0 ~\Rightarrow~ \sum_{j = 1}^n \alpha_j v_j = 0$ is trivially true.

But (as you can check) this is then also equivalent to the following

$\forall \alpha_j , \beta_j ~~\sum_{j = 1}^n \alpha_j v_j = \sum_{j = 1}^n \beta_j v_j ~~\Longleftrightarrow~~ \forall j ~~ \alpha_j = \beta_j$

which can be given a more intuitive interpretation. It basically says that two linear combinations (one with the $\alpha_j$'s and one with the $\beta_j$'s) of linearly independent vectors $v_j$ result in the same vector (i.e. $\sum \alpha_j v_j = \sum \beta_j v_j$), only when all the coefficients are already identical.

As you said, dependance is the negation of the above, and using our definition $(\ast)$ it would be

$\exists \alpha_j , \beta_j ~~\sum_{j = 1}^n \alpha_j v_j = \sum_{j = 1}^n \beta_j v_j ~~\text{and}~~ \exists j ~~ \alpha_j \neq \beta_j$

This says that the $v_j$ are dependant iff there are coefficients $\alpha_j, \beta_j$ which are not all the same (there is at least one $j$ such that $\alpha_j \neq \beta_j$), but still give the same linear combination $\sum \alpha_j v_j = \sum \beta_j v_j$.

---

tl;dr Independent vectors have unique linear combinations; Dependent vectors have ambiguous ones.