The comment about "pass[ing] off a full undefined integral as a solution to a problem" seems to capture the substance of the issue.
The FTC is being used here an a guarantee of existence for a certain (particularly simple) class of initial value problems,
$$
x' = f,\qquad x'(t_{0}) = x_{0},
$$
with $f$ an arbitrary but specified continuous function.
Suppose $f$ is a bounded function on some real interval $I$ containing $t_{0}$. For each real $t$ in $I$, the integral $\int_{t_{0}}^{t} f(s)\, ds$ of $f$ may be defined using only lower and upper sums of $f$. If in addition $f$ is continuous, the integral may be proven to exist. (This is usually proven by showing $f$ is uniformly continuous, and then deducing the lower and upper sums may be made as close as we like with a sufficiently fine partition.)
Separately (by the FTC), if $f$ is continuous, then the real-valued function $F$ defined by
$$
F(t) = x_{0} + \int_{t_{0}}^{t} f(s)\, ds
$$
solves the initial value problem above. Again, one part of the non-trivial content is the existence of a solution, particularly that every continuous, real-valued function on $I$ has an antiderivative. It was not a priori obvious that our IVP has a solution at all, nor that every continuous function on an interval has an antiderivative. (If $f$ has removable and/or jump discontinuities, for example, this IVP does not have solutions in this "classical" sense.)
It's true that the integral representation $F$ is not of itself "explicit," but mathematically it still represents a substantial guarantee.
an arbitrary exponential constant
? There are no constants except $C$. – Yuri Ginsburg Jan 04 '22 at 07:12