There is one immediate solution that exploits the symmetry as follows : twice the average is the average sum of distances from the given point to a point on the circle and its diametrically opposed, and each such sum is $> 2$ (except perhaps one).
Another way: consider a bounded figure $F$ in a normed space. For a point $P$ in the space consider $f(P)$ the average of the distance from $P$ to a point on $F$ ( we assume a positive measure on $F$). It is easy to see that $f(\cdot)$ is a convex function, and strictly convex, if $\|\cdot\|$ is strictly convex. Moreover, at $\infty$ the function $f$ becomes $+\infty$. Therefore, $f$ achieves a minimum at a point in the space. This point is unique if $f$ is strictly convex.
Now, assume moreover (on top of $f$ stricly convex) that $F$ is symmetric with respect to a point $O$, and the measure on $F$ is invariant under the symmetry through $O$. The the unique point of minimum of $f$ is also invariant under $O$, so it is $O$. We also see that $f$ is stricly increasing on every ray starting at $O$.
Note: this is the continuous analogue of finding a point whose sum of (weighted) distances to a family of points is smallest. If that set of points is invariant under some isometries, so is the point of minimum.
$\bf{Added:}$ @copper hat pointed out that it is not entirely obvious that there exists a point of minimum for the function $f$, if the ambient space is infinite dimensional ( Banach), something worth pondering on! However, if the problem has a center of symmetry $O$, then clearly $f(P) + f(P') \ge 2 f(O)$ ( perhaps strict), and moreover $f(P) = f(P')$, so no other argument is needed.