You are right. I used a table to make the problem more clear to me.
Let $x_i$ be the number of fishermen who caught i fishes. Suppose we have the following table:
$$\begin{array}{|c|c|c|c|c|} \hline i&1&2&3&4&\ldots&20 \\ \hline x_i&x_1&x_2&x_3&x_4&\ldots&x_{20} \\ \hline a_i&a_1&a_2&a_3&a_4&\ldots&a_{20} \\ \hline\end{array}$$
The number of fishes which are caught by fishermen who caught 1 fish is $1\cdot x_1=(a_1-a_2)\cdot 1$
The number of fishes which are caught by fishermen who caught 2 fishes is $2\cdot x_2=(a_2-a_3)\cdot 2$
The number of fishes which are caught by fishermen who caught 3 fishes is $3\cdot x_3=(a_3-a_4)\cdot 3$
$\ldots$
The number of fishes which are caught by fishermen who caught 20 fishes is $20\cdot x_{20}=a_{20}\cdot 20$
Summing up the terms
$$(a_1-a_2)\cdot 1+(a_2-a_3)\cdot 2+(a_3-a_4)\cdot 3+\ldots -a_{20}\cdot 19+a_{20}\cdot 20$$
This is a kind of a telescoping sum
$$a_1+(2\cdot a_2-a_2)+(3\cdot a_3-2\cdot a_3)+(4\cdot a_4-3\cdot a_4)+\ldots+(20a_{20}-19a_{20})$$
$$=a_1+a_2+a_3+\ldots+a_{20}$$