When a point "looks" in a direction perpendicular to the curved "mirror" it should "see" it's image directly in front of it, at a distance twice the distance to the "mirror" in that direction.
More formally, given a point $p$ we would like to reflect, any point $q$ on the reflection curve $\Gamma$ for which our given point has a perpendicular displacement $q-p \perp T_\Gamma(q)$ to should reflect $p$ to $p + 2(q-p)$ which is equal to $2q-p$.
$$ R_\Gamma: p \mapsto \left\{ 2q-p \ \big | \ q\in\Gamma, \ q-p \perp T_\Gamma(q)\right\}$$
When $\Gamma$ is a line, there is a unique $q$ for each $p$ such that $q-p \perp T_\Gamma(q)$, so $R_\Gamma$ is a function. This is no longer true for general curves, hence the need for set notation.
To find that set, note that for a parameterization $\gamma(t)$ of $\Gamma$, we have $T_\Gamma(q)$ is parallel to $\gamma'(t)$ whenever $q = \gamma(t)$, so you can solve
$$ (\gamma(t)-p)\cdot\gamma'(t) = 0 $$
for $t$ in terms of $p$ and evaluate $\gamma$ there to get $q$. If we call the set of solutions $T$ then we have
$$R_\Gamma(p) = \left\{ 2\gamma(\tau(p)) - p \ \big | \ \tau \in T \right\}$$
As an example, take the parabola defined by $y = x^2$. We can parameterize this so that $\gamma(t) = (t,t^2)$ and $\Gamma = \gamma(\Bbb{R})$. Note that $\gamma'(t) = (1,2t)$. Now for a given $p = (p_x,p_y)\in\Bbb{R}$, we have
$$ (\gamma(t) - p)\cdot\gamma'(t) = (t-p_x,t^2-p_y)\cdot(1,2t) = t-p_x + 2t^3 - 2tp_y $$
In general, this vanishes for one or three (real) values of $t$, so we'll call the one guaranteed to be real $\tau_0(p)$ and the other two $\tau_+(p)$ and $\tau_-(p)$. Let $U$ be the region where $\tau_\pm\in\Bbb{R}$, so by the cubic discriminant
$$ U = \left\{(x,y)\in\Bbb{R}^2 \ \Bigg | \ 4 y \geq 3\sqrt[3]{2x^2} + 2 \right\} $$
and let $V = \Bbb{R}^2 - U$. Note that in $U$, we may have $\tau_+ = \tau_-$ or possibly $\tau_\pm = \tau_0$, while in $V$ we always have exactly one real root $\tau_0$ of multiplicity $1$.
Then, the restriction of $R_\Gamma$ to $V$ is a function, and
$$R_\Gamma|_V(p) = 2\gamma(\tau_0(p)) - p$$
Without the restriction, we don't have a function, but we do have
$$R_\Gamma(p) = \left\{ 2\gamma(\tau(p)) - p \ | \ \tau \in \{\tau_0,\tau_+,\tau_-\} \right\}$$
however writing these explicitly means writing $\tau(p)$ explicitly, which are the roots of a cubic polynomial, and therefore somewhat messy to write out.
If I pick a particularly easy to work with kind of point, where $p_x = 0$ and $p_y = v \leq \frac{1}{2}$, then we have $T = \{ 0 \}$, $\gamma(\tau_0((0,v))) = (0,0)$, and so
$$ R_\Gamma((0,v)) = 2(0,0) - (0,v) = (0,-v) $$
which is just reflection over the $y$-axis.
For a slightly more interesting one, we'll take $p_y = \frac{1}{2}$ and $p_x = 2u^3$ can be anything. Then, $T = \{ u \}$, $\gamma(\tau_0((2u^3,\frac{1}{2})) = (u,u^2)$, so then
$$ R_\Gamma\left(\left(2u^3,\frac{1}{2}\right)\right) = 2\left(u,u^2\right) - \left(2u^3,\frac{1}{2}\right) = \left(2u-2u^3,2u^2 - \frac{1}{2}\right) $$
Any more interesting than that and the formulas get too messy for me to want to work with, but the framework is all here, so you are welcome to do that yourself.
