October 14, 2015. This is with $$ \frac{x^2 + y^2}{xy - t} = q > 0, $$
which I believe to be the intent of the question.
THEOREM: $$ \color{red}{ q \leq (t+1)^2 + 1 } $$
I got some help from Gerry Myerson on MO to finish the thing.
https://mathoverflow.net/questions/220834/optimal-bound-in-diophantine-representation-question/220844#220844
As far as rapid computer computations, for a fixed $t,$ we can demand $1 \leq x \leq 4 t.$ For each $x,$ we can then demand $1 \leq y \leq x$ along with the very helpful $x y \leq 4 t.$ Having found an integer quotient $q,$ we then keep only those solutions with $2x \leq qy$ and $2y \leq qx.$
In particular, for $t=1$ we find $q=5,$ then for $t=2$ we find $q=4,10.$ In both cases we have $q \leq (t+1)^2 + 1.$ We continue with $t \geq 3.$
With $t \geq 3, $ we also have $t^2 \geq 3t > 3t - 1.$
We are able to demand $xy \leq 4t$ by taking a Hurwitz Grundlösung, that is $2x \leq qy$ and $2y \leq qx.$ Define $k = xy - t \geq 1.$
Now, $xy \leq 4t,$ then $k = xy - t \leq 3t,$ then $k-1 \leq 3t - 1.$ Reverse, $3t-1 \geq k-1.$ Since $t^2 > 3t - 1,$ we reach
$$ t^2 > k-1. $$
Next, $k \geq 1,$ so $(k-1) \geq 0.$ We therefore might get equality in
$$ (k-1)t^2 \geq (k-1)^2, $$
but only when $k=1.$
$$ 0 \geq t^2 - k t^2 + k^2 - 2 k + 1, $$
$$ k t^2 + 2 k \geq t^2 + k^2 + 1. $$ Divide by $k,$
$$ t^2 + 2 \geq \frac{t^2}{k} + k + \frac{1}{k}. $$ Add $2t,$
$$ t^2 +2t + 2 \geq \frac{t^2}{k} + 2 t + k + \frac{1}{k}, $$
with equality only when $k=1.$ Reverse,
$$ \frac{t^2}{k} + 2 t + k + \frac{1}{k} \leq t^2 +2t + 2 $$
with equality only when $k=1.$
Here is Gerry's best bit, this would not have occurred to me. Here we are back to considering all solutions $(x,y)$ and all $k=xy-t.$ Draw the graph of the quarter circle $x^2 + y^2 = k q.$ As $x,y \geq 1,$ there are boundary points at $(1, \sqrt{kq-1})$ and $( \sqrt{kq-1},1).$ The hyperbola $xy = \sqrt{kq-1}$ passes through both points, but in between stays within the quarter circle. It follows by convexity (or Lagrange multipliers again) that, along the circular arc,
$$ \color{blue}{ xy \geq \sqrt{kq-1}}. $$ But, of course, $x^2 + y^2 = k q = qxy - t q$ is equivalent to our original equation $x^2 - q x y + y^2 = -tq.$
We have
$$ -tq = x^2 - q x y + y^2 = (x^2 + y^2 ) - q x y = k q - q x y \leq kq - q \sqrt{kq-1}, $$ or
$$ -tq \leq kq - q \sqrt{kq-1}, $$
$$ -t \leq k - \sqrt{kq-1}, $$
$$ \sqrt{kq-1} \leq t + k, $$
$$ kq -1 \leq t^2 + 2k t + k^2, $$
$$ kq \leq t^2 + 2 kt + k^2 + 1, $$ divide by $k,$
$$ q \leq \frac{t^2}{k} + 2 t + k + \frac{1}{k}. $$
For $t \geq 3$ and a solution with $xy < 4t,$ we showed
$$ \frac{t^2}{k} + 2 t + k + \frac{1}{k} \leq t^2 +2t + 2 $$
with equality only when $k=1.$ For all solutions, Gerry showed
$$ q \leq \frac{t^2}{k} + 2 t + k + \frac{1}{k}. $$
Put these together, we get
$$ q \leq t^2 +2t + 2 $$
with equality only when $k=1,$ that is $xy = t+1.$
ADDENDUM, October 15. Here is another way to get Gerry's main observation, with $k = xy - t,$ that $xy \geq \sqrt{kq-1}.$ We have $x,y \geq 1$ and
$kq =x^2 + y^2 .$ So $kq \geq x^2 + 1$ and $kq -(x^2 + 1) \geq 0.$ We also have $x^2 - 1 \geq 0.$ Multiply,
$$ (x^2 - 1) kq - (x^4 - 1) \geq 0. $$
Next, $y^2 = kq - x^2,$ so $x^2 y^2 = kq x^2 - x^4.$ That is
$$ x^2 y^2 = (kq-1) + (x^2 - 1)kq - (x^4 - 1). $$ However,
$$ (x^2 - 1) kq - (x^4 - 1) \geq 0, $$ so
$$ x^2 y^2 \geq kq - 1, $$
$$ \color{blue}{ xy \geq \sqrt{kq-1}}. $$
