Suppose we just wanted to find any real $q$ which satisfied that. Well, all we'd have to do would be to choose some $q\in (p,\sqrt{2})$ and so the obvious choice would be to take the center of this interval and so set $q=\dfrac{\sqrt{2}-p}{2}$. The problem with this is that because $p$ is rational and $\sqrt{2}$ is irrational, we find that $q$ must be irrational.
So, we have to be a little more clever. We still want $q$ to be in the above stated range, because those values satisfy $q>p$ and $p^2<2$, but how do we guarantee that $q$ is also rational? Well, we need to find a rational number in the range $(0,\sqrt{2}-p)$ which we can add to $p$. Let's call that value $r$ for now. Well $r$ is going to be dependent on $p$ so lets write $r$ as a function $r(p)$ which we want $r(p)<\sqrt{2}-p$ in the interval $0<p<\sqrt{2}$. We want $r(p)$ to take rational values for all rational $p$ so we should write it as a rational function $$r(p)=\dfrac{s_1(p)}{s_2(p)}$$ where $s_i$ is a polynomial in $p$. Now, there are many polynomials we could take to be $s_1$ and $s_2$ so lets put in some simplifying conditions. We want $r(p)<\sqrt{2}-p$ and we note that the RHS has a root at $p=\sqrt{2}$ so lets make $r(p)$ also have a root there. We want $r(\sqrt{2})=0$ which happens precisely when $s_1(\sqrt{2})=0$. Probably the most obvious choice for $s_1(p)$ then would be $s_1(p)=2-p^2$.
Now we need to try and find an $s_2(p)$ which makes $r(p)$ small enough. As we're trying to make $r(p)$ smaller than a linear function, let's make $r(p)$ grow like a linear function away from its singularities, so let's make the denominator linear and set $s_2(p)=ap+b$. So, we want $\frac{-b}{a}$ to be outside of the interval $(0,\sqrt{2})$ and we also want $s_2(p)$ to be large enough to satisfy the condition we impose on $r(p)$. At this point, many values of $a$ and $b$ will work and I can only assume Rudin chooses $a=1$ and $b=2$ to make the algebra a little neater later on.
Given this, we get $$r(p)=\dfrac{2-p^2}{p+2}$$ and after this we need to check that that this choice of $r(p)$ really satisfies the conditions, which it does.
I'll admit, this 'method' of finding a good $q$ involves a lot of trial and error, and some arbitrary choices, but this is because there are many possible $q$ that we could have used and the dependance of $q$ on $p$ just makes things that little bit more complicated. I can offer no advice on how to 'get a feel' for making these choices other than practice and trying to make 'reasonable' choices along the way.