Let $G$ be a group and let $S$ be a torsor over $G$; this is a set on which $G$ acts in a way that is (non-canonically) isomorphic to the action of $G$ by left multiplication on itself, and is a natural generalization of an affine space. For example:
- $n$-dimensional Euclidean space is a torsor over $\mathbb{R}^n$.
- The set of total orders on a set of size $n$ is a torsor over the symmetric group $S_n$.
- The set of possible locations of a bug on some vertex of an $n$-gon is a torsor over the cyclic group $\mathbb{Z}/n\mathbb{Z}$.
Then $S$ naturally inherits $G$ as a group of automorphisms (as a set). Now let $H$ be a subgroup of $\text{Aut}(G)$. Then picking an element $s \in S$ also allows us to identify $G$ with $S$ (via the map $g \mapsto gs$), hence defines an action of $H$ on $S$, but this action depends on the choice of $s$.
The subgroup of $\text{Aut}(S)$ (again, as a set) generated by $G$ and $H$ above is the semidirect product $G \rtimes H$ (regardless of the choice of $s$; exercise). Note the geometric significance of the case $G = \mathbb{R}^n, H = \text{O}(n)$.
(Of course in general the homomorphism $H \to \text{Aut}(G)$ is not injective, but I think the geometric significance is clearest when it is. In general perhaps a more algebraic perspective is appropriate; look up "split exact sequence.")