Let's look at the same question one dimension lower:
Say our world is actually 3-dimensional, and I have a cube sitting on my table. To a being only capable of perceiving 2 dimensions, what would it actually look like?
The immediately obvious question is: What do you mean with "a being only capable of 2 dimensions"?
In a sense we are such beings: Our retina is two-dimensional. We can reconstruct some 3D from having two eyes, but we don't have true 3D vision (we don't see the inside of the cube, for example). But what we see are 2D projections of the 3D objects (and our brain then reconstructs the third dimension from those two 2D images), and you already said that this is not what you want.
Another interpretation would be as in Flatland: The being lives in a plane in space, and only perceives whatever happens in that plane. If a 3D object passes through the plane, it simply sees (the border of) 2D sections of the cube. Philip Pugeau already provided a link showing several 3D sections of the 4D cube.
Yet another interpretation could be a flatlander that lives on the surface of the cube. The 3D analogue would be a 3D being (like us) living on the 3D surface of the tesseract. Such a being would see a quite strange geometry.
Let's first look at the cube-dwelling flatlander to get an intuition. Obviously, on the faces of the cube, everything is perfectly normal: The sides are subsets of Euclidean planes. Also, on the edges nothing special happens: While for us, the surface changes direction, for the flatlander, this would be invisible. His space would simply continue on the edge. Think of what happens when you open the cube to its net, like this:

The non-sliced squares just get squares located side by side in the Euclidean plane. Note that you can chose any of the edges to be among the uncut, as any two edges of the cube are equivalent. However the corners of the cube would be strange for him: When going round a corner, you get back to the original position already after $3/4$ of a full turn. So for the cube-dwelling flatlander, space would look like an "almost Euclidean" space with a regular lattice of "strange points".
Now let's look at the net of a tesseract:

Again, obviously on the surface cubes everything behaves just like in normal space. Also the squares where those cubes meet are quite normal. But at the edges, you'd get the same strange behaviour the cube-dwelling flatlander would observe at the cube corners, except that now we have complete lines of those. And of course on the corners, three such lines meet, and the behaviour gets even stranger. So we have a lattice of lines with strange behaviour.
Also be aware that those lines would not appear as physical things to the tesseract dwellers, but just as places in space where geometry goes crazy.