2

Question

When training and using an OCR algorithm for handwriting recognition, is it helpful to indicate the author of the handwriting?

Use Case

Have a warehouse full of documents that need to be transcribed into digital format.

We'd like to feed the documents into an OCR algorithm first. If the OCR algorithm reports a low confidence score, then we will pass the documents off to a real person for transcription - and of course use the results for additional algorithm training - so that future documents, especially by the same author, will have a higher chance of being transcribed at a satisfactory confidence level.

For each document, it is feasible but non-trivial, to determine the author of the document and give that to the OCR algorithm as well. We anticipate there should be on the order of 100 authors for 8 million documents.

Intuitively, I assumed that knowing the author would increase the effectiveness of the algorithm, but on further reflection, I am unsure if this is the case. When I read handwriting I don't usually think about the author, but instead intuit how to decipher the handwriting based on the style.

Note: By effectiveness I mean primarily higher accuracy, and secondarily lower resource usage.

Community
  • 1
Chris Dutrow
  • 463
  • In theory yes. In pratice no. You will need to generate training sets for the algorithm, each having a lot - i really mean -> a lot - of samples for the OCR engine to learn the patterns for each set. So only if you have a lot of samples of each person and time to train the OCR for each set. And let's not forget that the algorithm will have no idea of what is an author :) – linuxunil Jan 12 '17 at 00:28
  • @linuxunil - Thanks for your response! Do you have an idea on what order of size a training set would typically need to be? – Chris Dutrow Jan 12 '17 at 00:32
  • the key factor is how is you training data is different - sample by sample. Take an alphabet font. In the computer version an OCR can be easily trained to recognize an 'A' - in a few samples it will 'understand' the pixels patterns of what is an 'A'. Now try the same thing with handwriting... you will need a lot of samples to train the OCR that a random pattern of pixels is an 'A' in this persons letter. – linuxunil Jan 12 '17 at 00:57
  • now you know that every person writing is different - and even the same person write things different in different moments. This make the step of learning the particular person patterns a little tricky. – linuxunil Jan 12 '17 at 01:01
  • 1
    in a recent project i've used 1.5k of plate images - plate numbers only, same size, normalized colors and brightness - to train an OCR. And only got an 85% of accuracy. It's hard to guess what you will need... – linuxunil Jan 12 '17 at 01:05
  • 100 authors of 8 million handwritten documents? If I wrote that many, I don't think I'd be able to read my own handwriting. – JeffO Jan 12 '17 at 21:55

1 Answers1

2

It would helpful if the handwriting being recognized was written by the same person who created the writing for the OCR's training set. The OCR couldn't care less who that is, and in practice, you're not going to have the luxury of such a customized training sample.

Instead, the OCR is exposed to a wide variety of writing styles for training purposes, and then it picks out those features common to all handwriting styles for which certain letter and word features correspond to specific words and sentences.

Robert Harvey
  • 199,517
  • Thanks so much for your response. To clarify: In this use case, there will be a huge number of training documents, and a proportionately small number of authors. - I edited the question to make this more clear. – Chris Dutrow Jan 12 '17 at 00:15
  • I'll leave it as an open question whether or not 100 specific training sets is better than a single, large dataset. To find that out, you'd have to find an OCR that would let you change out the dataset on the fly. Matching the author with his own training set would almost certainly improve recognition, assuming you had sufficient training data for each set. Using a spelling and grammar checker after each recognition helps also. – Robert Harvey Jan 12 '17 at 00:20
  • Ah, ok. It sounds like one of my assumptions might be incorrect - which is that you can somehow indicate the author when feeding the image into the algorithm for use with a single data set. – Chris Dutrow Jan 12 '17 at 00:25