Dynamically generating and modifying non-Euclidean/Tardis houses while maintaining AI navigability

Question

General Description

I want to build a tech demo about a game concept I am carrying around for quite some time now. A lot of influences from different sources shaped this, and I will try to hint on its sources using parentheses in case some reader might want to look that up. Also, I will italic text that relates to my core concept, which together will lead to the actual question.

Imagine up to four players, which act as some form of state funded investigators, and a couple of AI agents as more or less spooky adversaries, investigating a case of missing people. They enter a two-story single family house with basement, which looks perfectly normal from the outside. From the inside, it should look perfectly normal too. At first. (The idea comes from the book House of Leaves and from the game Control)

Upon investigating the house, they should be able to find that one closet, which extends far more than it should be. Upon closing its door and reentering it, it will change its size and content a bit. If done long enough, it will open up to something, that should never fit inside such a building (yes, a bit of SCP here, and the title-giver, Dr. Who's phone both). But stays the same from the outside.

Anyway, the task would be to find somebody gone missing in there, and maybe add a couple of AI actors that actively try to interfere or just being plain spooky (again, SCP, but mainly Phasmophobia). And escort them back to the outside.

From my own gaming experience I can tell that even stuff like that can grow old fast after you learned to predict how these non-spaces do work. So that is where the procedural generation should come in. If the houses do look a bit alike on the inside, that is okay, but I strive to avoid using a template and only switching out parts (that is how Payday 2 handles its maps, by having certain places which rooms get switched out upon replaying).

Core Problems so far

• The outside must fit to the initial inside and not change during the game
• The inside must fit to the outside at the beginning
• Both should be mostly unique every play-through, with the option of using a seed to recreate these if needed (random room creation is handled here: Procedural... house with rooms generator)
• The inside should only chance if no one "is looking" (like that quantum stuff from Outer Wilds)...
• ...but if somebody is "inside" and not looking, they might disappear into an off-room or get stuck in a closet which doors lead somewhere else.
• But it must be avoided to trap the whole player squad in such a changing environment
• also, the AI actors should be able to traverse that building up to some point.... and... and... crap.

The one question to answer?

If you were observant, you'll have noticed that I stopped my core problems list at one point, which boils down to my core question:

-> How can I keep an AI actor able to traverse such a construct?

That is not only relevant for the opposing agents, but also that person the players would need to rescue. While I could have them getting carried back to the outside by the players, it would be preferred to have them walk back to the outside behind the players.

I am aware that the "generate stuff like described and keep it playable" is a whole theme of its own, but for these aspects I do have at least a plan or something to try out first. If that fails, I still can place a new question here, for example..

But all I know about AI movement boils down to: create your level, compile a navmesh, use an algorithm of choice (mostly A*) to make your bot move properly. But when I switch out parts of the level and the rooms inside can be bigger than the outside, I fail to imagine how to apply existing path-finding to that situation.

Even worse, as far as I remember, most of the navmesh solutions need to be done once, before the level is playable. Now it is updating mid game, possible braking running path-finding tasks because somebody closed and opened a door. I am aware that real-time updating is a thing (just any voxel and build game should do it), but they too work with normal space.

I mean, if I imagine a navmesh, it is mostly a 2D plane, that is projected onto a 3D environment from above, with coordinates in two dimensions fitting to these of the 3D space. Now I have a chunk of space, that will violate the 2D mapping in such a way, that it taps into wormhole-theory or non-Euclidean spaces.

The engine-choice or ide/framework to go with is not fixed yet, but for the sake of "use something where most of the non-programming stuff is already available to some point", I first would try to use Unity... if that matters.

Answer 1

Non-euclidean space is actually not as much of a problem for route finding as you would believe. The reason is that most standard route finding algorithms (A* included) do not operate on a grid but on a graph.

A graph is a collection of nodes connected by edges which have a traversal cost. Those edges can have any cost and can connect any two nodes. That means a graph does not necessarily need to be possible in euclidean space. For example, this graph (representing a "tardis" with two entrances, a large space inside and a small space to walk around it) is perfectly possible:

So most route finding algorithms will work perfectly fine with scenarios where objects are bigger on the inside than on the outside. The heuristic function of A* might get a bit trickier to implement in such a scenario, because the usual assumption "when it reduces the distance to the destination it's more likely to be the right way" might be less useful in a case where measuring distance is no longer a trivial problem. But that function is really just a performance optimization, so even a suboptimal heuristic function won't break your pathfinding.

Regarding changing the map layout at runtime: This always requires to change the route finding graph. Usually by swapping out the old graph section with the new one. This invalidates all paths going through the swapped out graph, so they need to be recalculated. It might in some cases also be prudent to recalculate any other paths, because there might now be a shorter option through the new graph section.

Answer 2

Since you're working with houses, you can leverage the conventions of those structures to simplify the problem.

To a first approximation, a house is a collection of rooms connected by doorways and windows. We can model that as a graph, where each room is a node, and each doorway or window that might allow line of sight or traversal between rooms is an edge. The outdoors is one "root" room, onto which any exterior doors/windows connect (at least initially - maybe later you add a door that exits to a different exterior world...)

This gives you a graph that you can use directly for AI pathfinding. It does not matter whether the connections between those rooms defies any possible embedding in 3D Euclidean space: as long as you have a graph of connections, A* can find connected paths through that graph. Working at the granularity of rooms will also make these searches fast, since you probably only have on the order of dozens of rooms at a time with a handful of doors each, so searching the entire graph can be done quickly, and does not depend on the amount of area they occupy.

You can make your AI pathfinding easier if all rooms have a convex floor plan, and any obstacles are small. Then you don't need a navmesh at all: to traverse a room from wherever you happen to be to any possible exit is just a straight line path, with local avoidance sufficing to walk around a table or chair in the way. You can still have non-convex rooms by splitting them into multiple convex parts, treating the seam between them as if it were just a large doorway. This also makes it easier to change the unoccupied part of a room hidden behind a concave bend, because it's logically a separate piece from the occupied portion.

(If you need more complicated pathfinding within rooms, you can still use separate navmesh islands for each one, without concern whether they overlap/link up in 3D space. As long as an agent can navigate to Door A in Room X using Room X's navmesh, you can then teleport them to the matching Door B in Room Y, and they can continue their navigation in that room using Room Y's navmesh, even if there's no way the two could coexist in one space)

Within each room, space is Euclidean, and you can use frustum checks to test which open doors or windows are within the field of view of any agent. Any which are not can then be flagged as candidates for modifying the house unseen - deleting them, or re-linking them to a matching portal somewhere else. You can do a similar thing with blank patches of wall - any that are unobserved can become candidate locations for a new edge to attach.

Each time you perform one of these modifications, invalidate any active AI navigation paths that used one of the modified edges. Then run a new A* search from that agent to the original "outside" room. If the search fails, your last modification has stranded this agent in an island with no traversible connections back to the outside, and you can simply undo the modification. Try another one next frame. You can run a similar search from each player position to avoid trapping players, or detecting when a player has become trapped if you want to let it happen for a little while, before offering a chance at salvation.

You can also run a connected component search to identify any rooms that have become unreachable/invisible from any reachable room, and prune them to save resources, freeing up budget for new rooms.