C# with XNA
Since you're new to this, XNA would have the advantage of also being a very good place to get started. From creating a new project to displaying a 2D sprite on screen it takes 5 lines of code:
private Texture2D texture;
texture = Content.Load<Texture2D>("sprite");
spriteBatch.Begin();
spriteBatch.Draw(texture, Vector2.Zero, Color.White);
spriteBatch.End();
It doesn't get much easier than this. Also has the advantage of having loads of educational samples available for it.
The problem is your performance requirement. But I've managed to draw thousands of sprites before in XNA with no problem. However there was no gameplay code behind it, and my computer is high-end so results would be biased. But with careful optimization it would probably work out. So this is my recommendation.
C# with SharpDX or SlimDX
Still staying in the managed C# realm, you could also go back one step and consider using SharpDX which is a new managed wrapper around DirectX, and they claim it to be about around 6 times faster than XNA. That should have enough power for your needs. And it's cross-platform.
There's also SlimDX to consider.
C++ with DirectX/OpenGL
And if you keep going further in that direction, you'll finally reach the pure C++ with DirectX/OpenGL realm. If you decide to go down that route (which by the way, will be much harder to deal with at this stage), there's a few things you can do to make your life easier.
For instance, you could use SDL to create the window for you and take care of input, and FMOD to handle the audio, both of which I have found much easier to work with than their native counterparts.
I also don't know what it's like nowadays, but the last time I worked with these, I found it was pretty easy to do 2D stuff in DirectX thanks to the D3DXSprite class which IIRC was quite similar to XNA's SpriteBatch, while with OpenGL I had to implement everything from scratch (i.e. messing around with textured quads, orthographic projections and implementing the batching).
Platform is PC
– Grungetastic Dec 30 '11 at 10:29