Application of extended Euclidean algorithm

Question

I tried applying the algorithm from Wikipedia in order to calculate $-133^{-1}\mod 256$

I spent already time myself finding the mistake but no success. This is how I did go about applying the algorithm:

r	newr	quotient	t	newt
256	-133	a	0	1
-133	123	-1	1	1
123	-10	-1	1	2
-10	3	-12	2	25
3	-1	-3	25	77
-1	0	-3	77	256

The solution should be $-77$

Answer 1

The loop invariant is $\,-133\, t_k\equiv r_k\pmod{\!256},\,$ so at termination, reading that in the 2nd last row this becomes: $\, -133\cdot 77\equiv -1,\,$ so negating $\,-133(-77)\equiv 1,$ i.e. $\,(-133)^{-1}\equiv -77$.

The Wikipedia algorithm is working with the system of least nonnegative residues, so you could have avoided the need for negation by working in this system, starting with $\,-133\equiv 123$.

Does this mean I could apply the algorithm for negative numbers if I set t=t∗r at the end?

The key idea of the algorithm is that $\bmod 256,\,$ given $\,r > r'> 0\,$ both multiples of $-133,\,$ then we can find a smaller multiple $\,r'' := r\bmod r' = r-q\,r'.\,$ Indeed if $\,r \equiv -133t,$ $\,r'\equiv -133t'$ then $\,r'' = r-q\,r' \equiv -133(t-qt')$ remains a multiple of $\,-133$.

The algorithm starts with the two obvious multiples $\,r_0\equiv 0\equiv 256,\,$ and $\,r_1 \equiv -133,\,$ then generates a chain of descending multiples till it reaches $\,0.\,$ The row with the last nonzero $\,r\,$ yields the sought result, as explained in the first paragraph above.

If you use negative residues $\,r_i\,$ then we can achieve descent by ensuring that they have decreasing magnitude, i.e. $\,|r_{k+1}| < |r_k|.\,$ Then the algorithm terminates with the least magnitude multiple $\,r\not\equiv 0\,$ and the gcd is $\,|r|,\,$ and the inverse can be obtained by negating (if need be), as above.

To gain better intuition on this Euclidean descent process (sequence of decreasing multiples) you may find it helpful to view the simpler case of Gauss's algorithm.

Answer 2

It is customary to agree on the order of operations in ordinary arithmetic and

$\quad −133^{-1} \text{ is interpreted to be equal to } (-1)\times 133^{-1}$

but in this case since

$\quad (−133)^{-1} = \bigr((-1)(133)\bigr)^{-1} = (-1)^{-1}(133)^{-1} = (-1)\times 133^{-1}$

you can't go wrong about what needs to be calculated.

On reading the OP's wikipedia article link, you'll find the following paragraph right before the algorithm that is specified using pseudocode:

To adapt the extended Euclidean algorithm to this problem, one should remark that the Bézout coefficient of $n$ is not needed, and thus does not need to be computed. Also, for getting a result which is positive and lower than $n$, one may use the fact that the integer $t$ provided by the algorithm satisfies $|t| \lt n$. That is, if $t < 0$, one must add $n$ to it at the end. This results in the pseudocode, in which the input $n$ is an integer larger than $1$.

Since the plan is to follow the specification exactly and feed the algorithm (routine) positive numbers, we can either get the answer this way (using a pre-process calculation),

$\quad \text{inverse}(123,256) \quad \text{(and getting a positive (residue) answer)}$

or this way (using a post-process calculation),

$\quad (-1) \times \text{inverse}(133, 256) \quad \text{(and getting a negative (residue) answer)}$

Here we'll go with the former method, setting $a = 123$ and $n =256$.

Here is the wikipedia pseudocode:

function inverse(a, n)
    t := 0;     newt := 1
    r := n;     newr := a
while newr ≠ 0 do
    quotient := r div newr
    (t, newt) := (newt, t − quotient × newt) 
    (r, newr) := (newr, r − quotient × newr)

if r > 1 then
    return "a is not invertible"
if t < 0 then
    t := t + n

return t

We can perform our calculations using the available algorithm calculation table
from the first example given in the wiki article,

Index $i$	Quotient $q_{i-1}$	Remainder $r_{i}$	$s_i$	$t_i$
0		256	n/a	0
1		123	n/a	1
2	2	10	n/a	-2
3	12	3	n/a	25
4	3	1	n/a	-77
5	3	0	n/a	256

Observe that the while loop in the pseudocode executes $4$ times, corresponding to

$\quad 2 \le \text{Index-}i \le 5$

in the table. Also, as discussed in the wiki article, we can 'turn off' the $s_i$ calculations.

So we're now 'out of' the while loop with

$\quad r = 1$
$\quad t = -77$
$\quad n = 256$

and resuming execution flow at

if r > 1 then
    return "a is not invertible"
if t < 0 then
    t := t + n
return t

The specified algorithm returns the value $179$, and since it is numeric (no error messages) we know that

$\quad 179 \cdot 123 \equiv 1 \pmod{256}$

---

Using the OP's calculation table,

r	newr	quotient	t	newt
256	123		0	1
123	10	2	1	-2
10	3	12	-2	25
3	1	3	25	-77
1	0	3	-77	256