I've always been confused as to when a quantum algorithm is allowed to have an oracle and what kind of a function the oracle can have. For instance, I know in Hamiltonian simulation algorithms, you need an oracle $O_H$ to access the matrix elements of the Hamiltonian $H$. This seems fine to me since a general Hamiltonian simulation algorithm doesn't know of the specific structure of the Hamiltonian it's trying to simulate, so it delegates that knowledge to the oracle.
On the other hand, in Grover's algorithm, the oracle applies an $e^{i\pi}$ phase to the state if it's a solution state, and doesn't do anything otherwise. This has always been sketchy to me since it seems like the oracle already knows what the solution state is, so how could you even construct it in the first place?
I'm trying to understand when it is justified to have an oracle in an algorithm, and how can we tell if a potential oracle can/cannot be constructed? Any insight would be greatly appreciated!