On the proximity of $a\sqrt b+b\sqrt a$ to an integer

Question

Let $\Sigma$ denote all ordered pairs $(a,b)$ of positive, square-free integers with $a> b$. What is the infimum of all $\kappa$ such that $$\left\{(a,b) \in \Sigma:a\sqrt b+b\sqrt a\,\,\text{is within}\,\,\frac1{(ab)^{\kappa}}\,\,\text{of an integer}\right\}$$ has finite cardinality?

Note that if $x$ has irrationality measure $\mu$ then $\mu$ is the infimum such that $$0<\left|x-\frac pq\right|<\frac1{q^{\mu}}$$ has finitely many solutions for $p,q$ integers. Here, the problem can be rephrased as determining $\kappa$ such that, $$0<\left|a\sqrt b+b\sqrt a-K\right|<\frac1{(ab)^\kappa}$$ has finitely many solutions for an integer $K(a,b)$ which is either $\lfloor a\sqrt b+b\sqrt a\rfloor$ or $\lceil a\sqrt b+b\sqrt a\rceil$. This formulation is close to the definition of an irrationality measure (which is $2$ in this case), but is not directly related since the parameters $p,q$ cannot be matched.

From empirical results, I believe that $\kappa\in[1,2]$, as letting $\kappa=2$ yielded no solutions for a long time. The code in PARI/GP is

squar(k)=for(a=2,+oo,for(b=2,a-1,if((issquare(a)==0 && frac(a*sqrt(b)+b*sqrt(a))<1/((a*b)^k)) || 1-frac(a*sqrt(b)+b*sqrt(a))<1/((a*b)^k),print1([a,b]," "))))

While I recognise that the question posed at the beginning of this post is extremely difficult to determine exactly, I would appreciate proofs that $\kappa>1$ or $\kappa<2$ should they be true.

_{Interestingly, when $a=b$, I haven't managed to find any solutions when $\kappa=1$. In fact, in this case, I conjecture that $\kappa\in[1/2,1]$.}

Answer 1

The answer to your problem is between $\frac{1}{2}$ and $3$.

Upper bound of $3$:

Suppose $a\sqrt{b}+b\sqrt{a} = m+\delta$ for $0 < \delta \le \dfrac1{(ab)^{3+\epsilon}}$ (the argument about to come works also for $m+(1-\delta)$ with $0 < \delta \le \dfrac1{(ab)^{3+\epsilon}}$). We can of course assume $m$ is large. Note $$a^2b+ab^2+2(ab)^{3/2} = m^2+2\delta m +\delta^2\tag{$\ast$},$$ and so $$4(ab)^3 = (m^2-a^2b-ab^2)^2+2(m^2-a^2b-ab^2)(2\delta m+\delta^2)+(2\delta m+\delta^2)^2\tag{$\ast\ast$}.$$ It's clear from $(*)$ that $m^2-a^2b-ab^2 \ge 0$ since $\delta$ is small. Therefore, $(**)$ implies $$2(m^2-a^2b-ab^2)(2\delta m+\delta^2)+(2\delta m+\delta^2)^2 \ge 1$$ (since $\delta \ne 0$, since $a,b$ are square-free). A little bit of algebra shows $\delta^2+2m\delta \ge y$ where $y := \sqrt{(m^2-a^2b-ab^2)^2+1}-(m^2-a^2b-ab^2)$. Some more algebra shows $$\delta \ge \sqrt{m^2+y}-m = \frac{y}{\sqrt{m^2+y}+m}.$$ Note $$y = \frac{1}{\sqrt{(m^2-a^2b-ab^2)^2+1}+(m^2-a^2b-ab^2)} \ge \frac{1}{2m^2},$$ so $\delta \ge \dfrac{1}{8m^3}$ for $m$ large enough.

This implies $\dfrac{1}{(ab)^{3+\epsilon}} \ge \dfrac{1}{8m^3} \ge \dfrac{1}{8a^3}$, a contradiction for $a$ large.

Lower bound of $\frac{1}{2}$:

Fix $\delta > 0$. We show that there are infinitely many pairs $(a,b) \in \Sigma$ such that $a\sqrt{b}+b\sqrt{a}$ is within $\dfrac{1}{(ab)^{1/2-\delta}}$ of an integer.

There are infinitely many $m$ for which $m^2+1$ is squarefree. See here. Letting $a = m^2+1$ for such $m$ and $b = 1$, we get $$\{a\sqrt{b}+b\sqrt{a}\} = \{\sqrt{a}\} = \sqrt{m^2+1}-m = \frac{1}{\sqrt{m^2+1}+m} \le \frac{1}{m} \le \frac{1}{a^{1/2-\delta}},$$ the last inequality holding for $a$ large enough.