This answer says that some models describes the universe as finite. How do those models describe the universe's border?
Does the border (theoretically) exist? Is it a solid border? Do they predict what happens if matter crosses this border (maybe it simply can't)?
Your intuition is correct. It does not really make any sense for the universe to have a border. Would it be an unbreakable wall? What would be on the other side? Models with a finite universe would not have a border. The universe would be curved into a closed loop like the surface of a sphere. In other words, if you go far enough in any one direction you come back to the starting point.
Take a sheet of paper. Glue/tape/whatever pairs of opposing edges together without twisting. The first such gluing will give a cylinder. The second then gives you a torus (a doughnut). This is a closed 2-d surface. It has no border. Things can just loop back to where they started by heading in a straight line.
Practical note: you're likely to get a bit of a sharp bend in your torus, but that's the fault of the paper.
In fact, try drawing curves on the paper to see what happens. If it's unfolded, connect points on borders exactly opposite each other (straight up/down or left/right on the paper). Fold it into a torus, and what do you see?
When folded draw some lines on the torus that pass through the edges of the paper. As curvy as you want. Labelling each edge entry and exit will help in the next part. Now unfold it and what do you see?
Indeed, if you've ever played the old arcade game Asteroids then you are already familiar with life on a torus. The lines you're drawing will behave exactly as your ship in the game will when you think of the unfolded paper as the screen.
This is basically what is meant by a closed universe. A more precise description requires the use of topology and the concept of a (smooth) manifold. Manifolds can have boundaries (the ends of the cylinder, for example), though as far as I know all serious cosmological models are boundaryless. There would be interesting and distinctive physics on a boundary, to the point that it should have been pretty obvious if one existed in the visible universe.
The models we're talking about are in the Friedmann–Robsetson–Walker family of solutions of general relativity, which are used because they have the properties of being spatially homogeneous, i.e. they are the same at every point in space, and also isotropic, i.e. every direction is equivalent.
There are many possible finite yet boundaryless spatial geometries, and I recommend zibadawa timmy's answer for a simple illustration of a toroidal two-dimensional universe and the accompanying analogy of the classic arcade game Asteroids. It is probably the most straightforward illustration that being finite does not imply having a border while at the same time making it intuitive that we don't need to actually consider a higher-dimensional space to embed it it in, because the rules of Asteroids don't actually need an extra dimensions. Such embeddings are just convenient visualizations.
However, where the toroidal universe differs from the FRW models is that it's not isotropic. You can see this by the fact that on a torus, going in some directions will get you back where you started, whereas going in others will wind you endlessly along the torus, never quite making it exactly where you've started at. Thus, not all directions behave the same way.
There are only four kinds of spatial geometries for the three-dimensional space of our universe that are homogeneous and isotropic: the Euclidean space $\mathbf{E}^3$, the hyperbolic space $\mathbf{H}^3$, the sphere $\mathbf{S}^3$, and the projective real space $\mathbf{RP}^3$, the last of which is like a sphere but with a different global topology.
So it's some kind of 4 dimension sphere (sphere in any direction you go in 3D) if the universe is finite?
A three-dimensional sphere, actually. Probably the most important intuitive leap here is that any particular embedding, or even whether there exists any embedding, is completely irrelevant. The surface of an ordinary beach-ball is actually a two-dimensional sphere, and as a manifold it makes sense whether or not we think of it a surface of some three-dimensional object. When we're talking about the universe, an embedding is a tool for visualization, not reality.
The universe could be a $3$-sphere, which we could think of as the surface of a ball in flat four-dimensional space, but there is no need to. That four-dimensional space wouldn't be part of our universe (not in general relativity, anyway), and such embeddings aren't unique. Actually, for completely general spacetime manifolds (i.e. if we try to embed spacetime in flat higher-dimensional spacetime), they're not even guaranteed to exist, which is a good thing because we don't need them--all our measurements are within the universe.
