To answer this question, we need to understand what is meant by:
the field $\operatorname{GF}(2^8)$ generated by the irreducible polynomial $m(x) = (\mathtt{0x11B})$.
and I'll try a gentle, progressive introduction to that.
One simple way to see this field is that as the set of the 256 octets $\{\mathtt{0x00},\mathtt{0x01},\dots,\mathtt{0xFF}\}$; with additive law bitwise XOR $\oplus\;$; and multiplicative law the commutative operation $*$ defined in either one of the following ways:
- By having $\mathtt{0x01}$ as the neutral element, $\mathtt{0x02}*\mathtt{0x80}=\mathtt{0x1B}\;$ (that's the given $\mathtt{0x11B}$ truncated to its low-order 8 bits), and the more intuitive $\mathtt{0x02}*\mathtt{0x02}=\mathtt{0x04}\;$, $\mathtt{0x02}*\mathtt{0x04}=\mathtt{0x08}\;$, $\mathtt{0x02}*\mathtt{0x08}=\mathtt{0x10}\;$, $\mathtt{0x02}*\mathtt{0x10}=\mathtt{0x20}\;$, $\mathtt{0x02}*\mathtt{0x20}=\mathtt{0x40}\;$, $\mathtt{0x02}*\mathtt{0x40}=\mathtt{0x80}\;$. These and the properties of $*$ (commutativity, associativity, distributivity with respect to $\oplus\;$) are enough to fully define $*$. The constant $\mathtt{0x1B}$ is such that every element except $\mathtt{0x00}$ (the neutral element of $\oplus\;$) has a multiplicative inverse, which can be found by inspection of the multiplication table (building this table, and deriving an algorithm which computes the product of two arbitrary elements without this table and at most 8 iterations, is left as a recommended exercise to the reader). This gives the result asked in the question, but not by the method thought, and is impractical by hand.
- Alternatively, we can define a commutative ring $(\mathbb N,\oplus,\otimes)$ where $\oplus$ is bitwise XOR, and $\otimes$ (sometime called carry-less multiplication) is the same as common multiplication when either argument is a power of two, and the associativity of $\otimes$ with respect to $\oplus$ is used to define the other cases. For example $\mathtt{0xDB}\otimes\mathtt{0x42}$ $=\mathtt{0xDB}\otimes(\mathtt{0x40}\oplus\mathtt{0x02})$ $=(\mathtt{0xDB}\otimes\mathtt{0x40})\oplus(\mathtt{0xDB}\otimes\mathtt{0x02})$ $=\mathtt{0x36C0}\oplus\mathtt{0x1B6}$ $=\mathtt{0x3776}$. We can define an analog to Euclidean division in $(\mathbb N,\oplus,\otimes)$, giving quotient $q$ and rest $r$ from dividend $d$ and non-zero divisor $n$, such that $d=(q\otimes n)\oplus r$, and the bit size of $r$ is less than the bit size of $n$. That allows us to define an analog in $(\mathbb N,\oplus,\otimes)$ of modular reduction in the ring $(\mathbb Z,+,\times)$. Our operation $*$ in $\operatorname{GF}(2^8)$ is $\otimes$, followed by reduction modulo $m=\mathtt{0x11B}$. That constant is such that $(\operatorname{GF}(2^8),\oplus,*)$ is a field, much like $(\mathbb Z_n,+,\times)$ is a field when $n$ is prime. Now we can apply the Extended Euclidean algorithm and answer the question by the method asked. We do as for computing an inverse modulo a positive integer, but use $\oplus$ instead of addition and subtraction, $\otimes$ instead of multiplication, and the analog of Euclidean division in $(\mathbb N,\oplus,\otimes)$.
An equivalent way to see the field $\operatorname{GF}(2^8)$ is as the set of the 256 polynomials with one variable, of degree less than 8, having bit coefficients per Boolean algebra; additive law the addition of polynomials; and multiplicative law the multiplication of polynomials followed by modular polynomial reduction of the product by the polynomial $m(x)=x^8+x^4+x^3+x+1$ (corresponding to bits in $\mathtt{0x11B}\;$). A necessary and sufficient condition for every element except zero being invertible, is that $m(x)$ is irreducible, which holds. Now we can apply the Extended Euclidean algorithm and answer the question by the method asked. We do as for computing an inverse modulo a positive integer, but use $m(x)$ as the modulus, and division of polynomials with bit coefficients instead of Euclidean division.
For the inversion itself when using the Extended Euclidean algorithm (one of many possibilities), a recommendable algorithm is this one.
TODO: add a description of the above