My understanding of your approach (correct me if I'm wrong) is that you want to compute the average distance between two points in the unit square by the following two-step process:
- Compute the average distance between two arbitrary perimeter points on the square.
- Compute the average distance between two arbitrary points lying on a line segment that is the length you computed in step 1.
Your reasoning is that any two points in the square determine a unique line segment between perimeter points, so by averaging over all those line segments, we should be able to get the answer. In fact, average distance between points on a line segment varies linearly with respect to the length of the line segment, so this in principle will get you a meaningful answer.
It's just not a meaningful answer to the question you're asking.
This will tell you the average distance between two points in the square chosen by first picking two perimeter points uniformly (that are not on the same side of the square), and then picking two points uniformly on the resulting line segment. But the key is this does not result in a uniform distribution of points on the square.
Put simply, your resulting pair of points will be more heavily weighted toward short line segments (e.g. a segment briefly crossing one corner) than if you chose the points uniformly. Two uniformly-chosen points in the square are more likely to be on a long perimeter line segment than a short one, precisely because such lines are longer.
Edit: For an illustration of how different your distribution is from uniform, let's calculate the odds that both points show up in the same triangular corner region, bounded by the segment that intersects with the perimeter 1/10 of the distance from the corner:

If we use a uniform distribution, this should only occur with probability $.02 \times .02 \times 1/4 = 0.0001$. Each point has a 2% chance of landing in one of these areas, and there is a $1/4$ chance they landed in the same area.
In your distribution, this occurs with probability $2/3 \times 0.1 \times 0.1 \approx 0.0067$. That is, there's a $2/3$ likelihood of picking two adjacent sides, followed by a $1/10$ chance for each point that it is close enough to the shared corner. This is $\approx 67$ times more likely than uniform.