Joachimsthal's notation
This "trick" to find the equation of the tangent to a conic section (ellipse, parabola, hyperbola) can be understood using the notation of Ferdinand Joachimsthal (1818-1861), a German geometer who had been a student of Kummer, Dirichlet, Steiner, Jacobi and Bessel — quite a list! We summarise some material here from Cut-The-Knot.
Consider the conic with equation $Ax^2 + 2Bxy + Cy^2 + 2Fx + 2Gy + H = 0$; writing the left-hand side as $s(x,y)$ this is just $s(x,y)=0$ or $s=0$ for short. Now let us denote
$$s_{ij}=Ax_ix_j + B(x_iy_j + x_jy_i) + Cy_iy_j + F(x_i + x_j) + G(y_i + y_j) + H$$
for two points $(x_i,y_i)$ and $(x_j,y_j)$ which do not necessarily lie on the conic. In particular, if the two points are identical we have
$$s_{ii} = Ax_i^2 + 2Bx_iy_i + Cy_i^2 + 2Fx_i + 2Gy_i + H = s(x_i, y_i)$$
so the condition that $(x_i,y_i)$ lies on the conic is just $s_{ii} = 0$.
Moreover, the line joining $(x_i,y_i)$ and $(x_j,y_j)$ is tangent to the conic if and only if $s_{ij}^2 = s_{ii}s_{jj}$.
Proof of tangency condition $s_{ij}^2 = s_{ii}s_{jj}$
A really nice way of showing this, using matrices, homogeneous coordinates, and ideas from projective geometry, is proved by Wilson Stothers here.
The approach taken at Cut-the-Knot uses less advanced concepts at the expense of nastier algebra and not quite dealing properly with "ideal points" ("points at infinity"). Write the vector equation of the line in parametric form, $(x,y)=t(x_i,y_i) + (1-t)(x_j,y_j)$. When the parameter $t=0$ we are at $(x_j,y_j)$ and when $t=1$ we are at $(x_i,y_i)$.
To find the line's intersection points with the conic, substitute $x = x_j + t(x_i - x_j)$ and $y = y_j + t(y_i - y_j)$ into our expression for $s(x,y)$, so we can express $s$ in terms of the parameter $t$. More neatly presented algebraic details are given in the answers to this question. It may be helpful to observe:
\begin{align}
s(x + \Delta x, y + \Delta y) - s(x, y) &= A(x + \Delta x)^2 - Ax^2 \\
&+ 2B(x + \Delta x)(y + \Delta y) - 2Bxy \\
&+ C(y + \Delta y)^2 - Cy^2 \\
&+ 2F(x + \Delta x) - 2Fx \\
&+ 2G(y + \Delta y) - 2Gy \\
&+ H - H \\
\implies s(x + \Delta x, y + \Delta y) - s(x, y) &= A (\Delta x)^2 + 2B \Delta x \Delta y + C (\Delta y)^2 \\
&+2(A x \Delta x + B x \Delta y + B y \Delta x + C y \Delta y + F \Delta x + G \Delta y)
\end{align}
If we write $s(t) = s(x_j + t(x_i - x_j), y_j + t(y_i - y_j))$, so $\Delta x = t(x_i - x_j)$, we obtain a quadratic in $t$,
\begin{align}
s(t) - s_{jj} &= t^2 \{ A (x_i - x_j)^2 + 2B (x_i - x_j)(y_i - y_j) + C (y_i - y_j)^2 \} \\
&+2t \{ A x_j (x_i - x_j) + B x_j (y_i - y_j) + B y_j (x_i - x_j) \\
&\quad \quad + C y_j (y_i - y_j) + F (x_i - x_j) + G (y_i - y_j) \}
\end{align}
With a bit of work, this can be expressed as
$$s(t) = (s_{ii} - 2s_{ij} + s_{jj})t^2 + 2(s_{ij} - s_{jj})t + s_{jj}$$
We can see $s_{ii} - s_{ij}$ is
$$Ax_i(x_i - x_j) + B x_i (y_i - y_j) + B (x_i - x_j) y_i + C y_i(y_i - y_j) + F(x_i - x_j) + G(y_i - y_j)$$
and, symmetrically, $s_{jj} - s_{ij}$ is
$$Ax_j(x_j - x_i) + B x_j (y_j - y_i) + B (x_j - x_i) y_j + C y_j(y_j - y_i) + F(x_j - x_i) + G(y_j - y_i)$$
where the brackets have subtractions in the reverse order, so opposite signs. This lets us sum the two expressions, e.g. noting $Ax_i(x_i - x_j) + Ax_j(x_j - x_i)$ is $Ax_i(x_i - x_j) - Ax_j(x_i - x_j)$ so factorises as $A(x_i - x_j)^2$, whereas e.g. $F(x_i - x_j)$ and $F(x_j - x_i)$ just cancel. Overall,
$$s_{ii} - 2s_{ij} + s_{jj} = A(x_i - x_j)^2 + 2B(x_i - x_j)(y_i - y_j) + C(y_i - y_j)^2 $$
which is our coefficient of $t^2$. Also, the curly bracket multiplying $2t$ looks like $s_{jj} - s_{ij}$ but the bracketed subtractions, and therefore the sign, are reversed, so the $t$ coefficient must be $2(s_{ij} - s_{jj})$.
This manipulation is easier if we write the conic in matrix form:
$$\begin{bmatrix}x & y & 1 \end{bmatrix} \begin{bmatrix}A & B & F \\ B & C & G \\ F & G & H \end{bmatrix} \begin{bmatrix}x \\ y \\ 1 \end{bmatrix} = 0$$
or simply $\mathbf{x'Mx} = 0$ where $\mathbf{M}$ is a symmetric three-by-three matrix. Then $s_{ij} = \mathbf{x}_i' \mathbf{Mx}_j$ and using the symmetry $s_{ij} = s_{ji}$ we have:
\begin{align}
s(t) &= (\mathbf{x}_j + t(\mathbf{x}_i - \mathbf{x}_j))' \mathbf{M} (\mathbf{x}_j + t(\mathbf{x}_i - \mathbf{x}_j)) \\
&= (\mathbf{x}_i - \mathbf{x}_j)' \mathbf{M} (\mathbf{x}_i - \mathbf{x}_j) t^2 + \left(\mathbf{x}_j' \mathbf{M} (\mathbf{x}_i - \mathbf{x}_j) + (\mathbf{x}_i - \mathbf{x}_j)' \mathbf{M} \mathbf{x}_j \right) t + \mathbf{x}_j' \mathbf{M} \mathbf{x}_j \\
&= \left( \mathbf{x}_i' \mathbf{M} \mathbf{x}_i - \mathbf{x}_i' \mathbf{M} \mathbf{x}_j - \mathbf{x}_j' \mathbf{M} \mathbf{x}_i + \mathbf{x}_j' \mathbf{M} \mathbf{x}_j \right) t^2 + \\
& \quad \left( \mathbf{x}_j' \mathbf{M} \mathbf{x}_i - 2\mathbf{x}_j' \mathbf{M} \mathbf{x}_j + \mathbf{x}_i' \mathbf{M} \mathbf{x}_j \right) t + \mathbf{x}_j' \mathbf{M} \mathbf{x}_j \\
&= (s_{ii} - s_{ij} - s_{ji} + s_{jj})t^2 + (s_{ji} - 2s_{jj} + s_{ij})t + s_{jj} \\
s(t) &= (s_{ii} - 2s_{ij} + s_{jj})t^2 + 2(s_{ij} - s_{jj})t + s_{jj}
\end{align}
At the point the line meets the conic, the equation of the conic must also be satisfied so $s(t)=0$, yielding a quadratic equation
$$(s_{ii} - 2s_{ij} + s_{jj})t^2 + 2(s_{ij} - s_{jj})t + s_{jj} = 0$$
The discriminant is
$$\Delta = 4(s_{ij}-s_{jj})^2 - 4(s_{ii}-2s_{ij}+s_{jj})s_{jj}$$
which simplifies to $4(s_{ij}^2 - s_{ii}s_{jj})$. For a tangent, we want the line to touch the conic only once, i.e. one solution to the quadratic, so the discriminant must be zero. This happens when $s_{ij}^2 = s_{ii}s_{jj}$.
Equation of tangent to given point on the conic
In particular, it's common to take $(x_i,y_i) = (x_1, y_1)$ as a point on the conic at which we want to find the tangent. Then the condition that $(x_j,y_j)$ is a point on the tangent to the conic at $(x_1,y_1)$ is just $s_{1j}^2 = s_{11}s_{jj}$ and since $s_{11}=0$ (the point of tangency is on the conic) this becomes $s_{1j}=0$. As $(x_j,y_j)$ represents a general point on the tangent line, it's standard to drop the suffix $j$ and call the point $(x,y)$ instead. Similarly we abbreviate $s_{1j}$ to $s_1$, to indicate the first point in $s_{ij}$ is the given point of tangency $(x_1,y_1)$ and the second point is kept general.
This is the well-known result that if we take the tangent to the conic $s=0$ at a point $(x_1,y_1)$ on the conic, then the equation of the tangent is $s_1 = 0$, i.e.
$$Ax_1x + B(x_1 y + x y_1) + Cy_1y + F(x_1 + x) + G(y_1 + y) + H = 0$$
Equations of tangents from given point outside the conic
Again consider point $i$ as a fixed point $(x_1, y_1)$ and point $j$ as a general point $(x, y)$ on the tangent line, but this time suppose $(x_1, y_1)$ is not on the conic. This means $s_{11} \neq 0$, so we cannot just write $s_1 = 0$ to find the tangent. Instead we will need to use the more general tangency condition, $$s_1^2 = s_{11}s$$
For example, suppose we seek the tangents to $2xy = 5$ that pass through $(x_1,y_1) = (6,-10)$. Then $s = 2xy-5$ and $s_{11}=2x_1 y_1-5 = -125$. We have $s_1 = x_1 y + x y_1 - 5$ so $s_1 = 6y - 10x - 5$. Our tangency condition is
\begin{align}
(6y - 10x - 5)^2 &= -125(2xy-5) \\
\implies 100x^2 - 120xy + 36y^2 + 100x - 60y + 25 &= -250xy + 625 \\
\implies 100x^2 + 130xy + 36y^2 + 100x - 60y - 600 &= 0 \\
\implies 50x^2 + 65xy + 18y^2 + 50x - 30y - 300 &= 0 \\
\implies (10x + 9y + 30)(5x + 2y - 10) &= 0
\end{align}
Hence our tangents are $10x + 9y + 30 = 0$ and $5x + 2y - 10 = 0$.
Length of tangent to a circle from a given point
Consider the circle with equation $s = x^2 + y^2 + 2Fx + 2Gy + H = 0$, so we have $A=C=1$ and $B=0$. Then our equation for the $t^2$ coefficient reduces to
$$s_{ii} - 2s_{ij} + s_{jj} = (x_i - x_j)^2 + (y_i - y_j)^2 $$
which by Pythagoras is the square of the distance between $(x_i, y_i)$ and $(x_j, y_j)$. If we take the first point as a given $(x_1, y_1)$ and the second point as the place the tangent from $(x_1, y_1)$ meets the circle, then $s_{jj}=0$ (as the given point is on the conic) and $s_{ij}=0$ (since by the tangency condition $s_{ij}^2 = s_{ii} s_{jj}$ but $s_{jj}=0$) and the formula simplifies further:
$$s_{11} = (x_1 - x_j)^2 + (y_1 - y_j)^2 $$
The length of the tangent from $(x_1, y_1)$ to the circle $s = 0$ is therefore just $\sqrt{s_{11}}$, i.e. $$\text{tangent length} = \sqrt{x_1^2 + y_1^2 + 2Fx_1 + 2Gy_1 + H}$$
The tangent length "trick" is often taught at the same time as the tangent equation "trick" (in my old textbook, this is the very next equation) but this time we see it's specific to the circle, not a general conic — see this question for the general case. Note that if $(x_1, y_1)$ lies on the circle, then $s_{11}=0$ so we obtain length $\sqrt{s_{11}}=0$ as expected. When $(x_1, y_1)$ is inside the circle, $s_{11} < 0$ and there's no real value for $\sqrt{s_{11}}$, but this is as expected since no tangent can be drawn through $(x_1, y_1)$. However, there is a geometric meaning to $s_{11}$ that applies to a point whether it is outside, on, or inside the circle: the power of a point with respect to the circle.
Poles and polars
You may have come across the notion of poles and polars in the context of circle inversion, but the idea generalises to other conic sections. Poles are points and polars are lines; the idea is that, with respect to a given conic section, each pole is associated with a unique polar and vice versa, by a transformation called a polarity. In fact the pole $(x_1, y_1)$ has polar $s_1 = 0$ with respect to the conic $s = 0$.
The "trick" that $s_1 = 0$ is the tangent to the point $(x_1, y_1)$ on the conic works because the polar of a point on the conic is the tangent at that point (and conversely, the pole of a tangent is its point of tangency).
What if $(x_1, y_1)$ is not on the conic? Suppose there are two tangents from $(x_1, y_1)$ to the conic, as in the hyperbola example above. (This isn't guaranteed; e.g. no tangent passes through the centre of a circle.) Let the tangents through $(x_1, y_1)$ meet the conic at $(x_2, y_2)$ and $(x_3, y_3)$. These points are on the conic, so $s_{22}=s_{33}=0$. The equations of the tangents are $s_2 = 0$ and $s_3 = 0$, so e.g. $s_{2j}=0$ if the point $(x_j,y_j)$ lies on the tangent. Since $(x_1, y_1)$ lies on both tangents, we have $s_{21}=s_{31}=0$. However, the definition of $s_{ij}$ was symmetric in $i$ and $j$, so we must also have $s_{12}=s_{13}=0$. This means both $(x_2, y_2)$ and $(x_3, y_3)$ satisfy $s_1 = 0$: in other words, the polar of $(x_1, y_1)$, with respect to the conic $s = 0$, is the line that passes through the two points where the tangents through $(x_1, y_1)$ meet the conic.
Similarly we can prove La Hire's theorem that if $(x_1, y_1)$ lies on the polar of $(x_2, y_2)$ with respect to the conic $s = 0$, then $(x_2, y_2)$ lies on the polar of $(x_1, y_1)$ with respect to the same conic. We are given that $(x_1, y_1)$ lies on the line $s_2 = 0$, so $s_{21}=0$. By symmetry $s_{12}=0$, so $(x_2, y_2)$ must satisfy $s_1 = 0$, so lies on the polar of $(x_1, y_1)$ as required.
