What are the things we need to take care for a web application to handle large number of requests(say 10000 simultaneous requests).
Increasing the number of servers and distributing the load is one way , but are there other ways or configurations that can be done on application server ?
Reasonable caching of data and pages, and optimization of high frequency queries. Those two alone will get you far (I am in charge of a service that sees 1 million requests for dynamic data per day, not enormous, but not tiny either).
One of the problems I see most often is people sticking to a normalized form for their databases, instead of building denormalized Star-style tables.
Normalization isn't always the best choice when you have a certain subset of your data that accounts for 99% of your data requests.
If you're talking 10,000 simultaneous requests, your whole data layer is going to be severely taxed. You definitely want that to be running as lean as possible.
The first thing I would do is do some load-testing. Start with a base load of say 500 users, then ramp up to your target of 10k users progressivly every few minutes.
While doing your load-testing you should monitor your test server and profile your application.
It is often hard, even when you have first-hand knowledge, of the application to make correct guesses about where to optimize. This why using tools is important.
If you are using VS 2010, in the Ultimate edition there is a load-testing and profiling tool that can simulate a variable number of users. Other tools from other vendors also exists. Such tools let you write scenarios for your virtual users to execute and execute them en masse according to some specs you have defined.
This will allow you to look at memory consumption of your application or profile your application and determine what is causing, if any, performance problem you might have when you are sollicited by a large number of users.
Resulting from those tests you might find that your application does not have a problem handling that many users or that it can't even handle half that load.
Depending on where the problem lie you can then investigate proper measures for this speicific area, whether it be the db, network, app code, server memory, etc.
You can use a content delivery network like http://www.akamai.com/ to off-load the bulk of the requests.
Of your 10,000 requests, 1 is for the page which you serve on your server and the other 20 or so are for images, css, javascript. By letting A CDN handle all of those requests you drastically reduce the load on your server. Now all you focus on is a small fraction of the traffic that is for your actual web application.
The advice at How do you off load work from the database? should give you a lot of material to think about.
Beyond that, pay attention to any standard platform specific optimizations. For instance for mod_perl applications you invariably want to have a reverse proxy in front of your website that serves static images, and which causes mod_perl processes to not waste time talking to slow clients.
You should also use standard load balancers, multiple web servers, etc.
Beyond that, be very careful to keep your architecture as simple and straightforward as possible. I've seen people create horribly complex systems with multiple layers of RPCs because they "need to scale" and then find that they need a ton of hardware because of RPC overhead. With a proper architecture, you should be able to have a site in the top 1000 busiest websites using only a handful of webservers. If your architecture can't do that, odds are very good that the problem is that you have a bad architecture, and not that you have too many users.
That said, if you've got a very, very complex website, with insane traffic, then you need to do something much more complex. You'll want to distribute everything. And you'll want to layer things with RPC calls to backend services that themselves do the same thing. The RPC calls should be as efficient as possible. (Google uses protobuf for this. DO NOT use XML. If you want human readable, use something like JSON. Trust me on this.) Furthermore you absolutely need to have very sophisticated monitoring on your systems. If pages get slow, you need to be able to track which RPC call 3 layers deep is slow. Furthermore if a particular RPC layer is getting overloaded or slow, you need tools to automatically detect this, and track down where the problem is coming from.
Trust me when I say that this is a complex architectural problem. (I've seen Google's infrastructure for handling this. It is a lot easier in theory than practice.) Please trust me further when I say that you really don't want to open this can of worms until after you've demonstrated a traffic volume where it makes sense. If you try, you're likely to wind up being Yet Another company patting yourself on the back for successfully delivering 300,000 hits/hour with 50 webservers, not realizing that 1 should be enough.
