12

Temporal anti aliasing (and other temporal algorithms) work by matching pixels this frame with pixels from the last frame and then using that information.

I get that you can use the last and current frame matrices along with motion vector information to match the pixels between frames.

What I don't get though is how do you know whether the reprojected pixel is valid or not? For instance the old pixel may now be hidden behind a different object.

Is it by color only? If so, how are animated textures or changing light conditions handled?

Alan Wolfe
  • 7,801
  • 3
  • 30
  • 76

2 Answers2

6

One strategy mentioned in Brian Karis's talk about TAA is neighborhood clamping. The general idea is that, for the previous frame's pixel to be valid, its color should be in the color range found in the neighborhood (say 3x3 pixels) of the current pixel this frame.

This rejects history from changing light conditions, which is probably what you want anyway if you don't want moving shadows to produce ghosting.

(Animated textures, depending on the speed of the animation, could also be handled with a motion vector, if you have a predictable UV mapping or can guess reasonably well.)

John Calsbeek
  • 3,972
  • 21
  • 31
  • 1
    Have you heard of anyone using the depth buffer value as a sanity check? – Alan Wolfe Aug 09 '15 at 14:16
  • @AlanWolfe No, and I think that is because the motion vector texture is usually 2-component: you'd need a change-in-Z component to know what the depth buffer value should be, and that's not nicely bounded by the screen size. I suspect you could get better rejection strategies than that by adding more per-pixel information. – John Calsbeek Aug 09 '15 at 16:48
  • Ah OK. What kind of information do you think would be helpful to add. Shading parameter type stuff to be able to tell if it's the same material? – Alan Wolfe Aug 09 '15 at 17:26
  • 1
    @AlanWolfe I don't have a lot of concrete ideas. I'm not an expert on when temporal reprojection with neighborhood clamping breaks down and produces artifacts and what information would be useful in those situations. Perhaps transparents (no motion vector information) combined with high-frequency lighting are producing artifacts, and you need some obscurance information. Perhaps geometric aliasing is your problem and you need some other information. – John Calsbeek Aug 09 '15 at 18:11
1

temporal reprojection works surprisingly well and simplistic with (in between) volume-marching (where rays scatter through a cloud of different density, with hard or smooth density-gradients)

validity is is then as simple as an upper bound difference between 2 euclidean-distances (currentFrame + previousFrame)

it may help, to use objectIDs to maintain sharper borders between any 2 solids, but too sharp borders will cause artefacts on movement. The definition of "solid" starts to get fluent, as soon as you do pure volume-marching, and then objectIDs may not be needed for reprojection (and only vor texturing), as in many more basic shaders of https://www.shadertoy.com/results?query=reproject&sort=newest&filter= explore my volume-marching with quaternion-camera unifications: https://www.shadertoy.com/view/7lBXWW (voumetric laser refraction on spheres+prism) https://www.shadertoy.com/view/wl3yDs (2 dynamic lights near a mandelbulb)

ollj
  • 21
  • 3