3

I'm a bit of an amateur when it comes to web development, but I need to do something a little more sophisticated than usual at work and I'm not sure how best to proceed.

The setup is as follows. I have a web server set up which takes some input from the user, gathers a reasonably large amount of data from various web APIs, does some processing, and sends some processed data back to the user. In particular, the server is stateless.

What I want to do next is allow the user to inspect the output and dynamically request additional information about the unprocessed data. I can think of two potentially sensible ways to do this:

  1. Give the user all of the data and let him or her sort it out.
  2. Store the unprocessed data in a database and retrieve it when asked.

I can rule out option 1 because some of the unprocessed data is proprietary - I am only allowed to distribute summary statistics. And I would rather not do option 2 because it seems like an ordeal and there is no need to keep the data around once the user is finished making requests. What I want is a third option which allows me to keep the data in the first request around for subsequent requests (until, say, the user is inactive for 5 minutes or something). Does this sort of system exist? Or is there a reason why it is a bad idea?

Paul Siegel
  • 213
  • 2
  • 6

1 Answers1

3

There's nothing particularly wrong with having a stateful communication rather than stateless. But it is harder to implement from both the client and server perspective.

In your case I think the answer will be determined by whether you are able to make the second 'additional' request without first making the first 'processing' request and how you handle this process.

I'm guessing that 'processing' is a computationally expensive process, which you don't want to repeat, so caching the result seems like a good idea in any case. What if the client sends the same process request twice for example? you would want to just return the result.

Once you have these intermediate results floating around, you could offer a list of possible 'additional' requests via a third 'list intermediate data' request. this would allow you to keep the service strictly speaking stateless, as you can make this 'list' request at any point and then follow up on one of its options without having to have made the first 'process' request.

The length of time you keep the intermediate data is a tricky one. But perhaps you can instead offer a 'delete intermediate data' service to the client, so they can tell you when they have finished.

Ewan
  • 75,506
  • The data I'm gathering before the processing comes from databases which update very quickly, and I want the user to expect that the processed output is up-to-date. The same user initial user input could yield dramatically different processed output five minutes later (the processing takes up to 20 seconds or so), but it may be important for the user to get some supplementary statistics (very cheap after the processing) in those five minutes. This is why I don't think a database is appropriate for my use-case, but maybe there are some lightweight caching frameworks which are appropriate? – Paul Siegel Oct 12 '15 at 01:39