GNU Ocrad Manual

This manual is for GNU Ocrad (version 0.25, 31 March 2015).

• Introduction: Purpose and features of GNU Ocrad
• Character sets: Input charsets and output formats
• Invoking ocrad: Command line interface
• Filters: Postprocessing the produced text
• Library version: Checking library version
• Library functions: Descriptions of the library functions
• Library error codes: Meaning of codes returned by functions
• Image format conversion: How to convert other formats to pnm
• Algorithm: How ocrad does its job
• OCR results file: Description of the ORF file format
• Problems: Reporting bugs
• Concept index: Index of concepts

This manual is free documentation: you have unlimited permission to copy, distribute and modify it.

1 Introduction

GNU Ocrad is an OCR (Optical Character Recognition) program and library based on a feature extraction method. It reads images in pbm (bitmap), pgm (greyscale) or ppm (color) formats and produces text in byte (8-bit) or UTF-8 formats. The pbm, pgm and ppm formats are collectively known as pnm.

Ocrad includes a layout analyser able to separate the columns or blocks of text normally found on printed pages.

For best results the characters should be at least 20 pixels high. If they are smaller, try the '--scale' option. Scanning the image at 300 dpi usually produces a character size good enough for ocrad.

2 Character sets

The character set internally used by ocrad is ISO 10646, also known as UCS (Universal Character Set), which can represent over two thousand million characters (2^31).

As it is unpractical to try to recognize one among so many different characters, you can tell ocrad what character sets to recognize. You do this with the '--charset' option.

If the input page contains characters from only one character set, say 'ISO-8859-15', you can use the default 'byte' output format. But in a page with 'ISO-8859-9' and 'ISO-8859-15' characters, you can't tell if a code of 0xFD represents a 'latin small letter i dotless' or a 'latin small letter y with acute'. You should use '--format=utf8' instead.
Of course, you may request UTF-8 output in any case.

3 Invoking ocrad

The format for running ocrad is:

     ocrad [options] [files]

Ocrad supports the following options:

'-h'

'--help'

Print an informative help message describing the options and exit. 'ocrad --verbose --help' describes also hidden options.

'-V'

'--version'

Print the version number of ocrad on the standard output and exit.

'-a'

'--append'

Append generated text to the output file instead of overwriting it.

'-c name'

'--charset=name'

Enable recognition of the characters belonging to the given character set. You can repeat this option multiple times with different names for processing a page with characters from different character sets.
If no charset is specified, 'iso-8859-15' (latin9) is assumed.
Try '--charset=help' for a list of valid charset names.

'-e name'

'--filter=name'

Pass the output text through the given built-in postprocessing filter (see Filters). Several filters can be applied in sequence using as many '--filter' and '--user-filter' options as needed. The filters are applied in the order they appear on the command line.
Try '--filter=help' for a list of valid filter names.

'-E file'

'--user-filter=file'

Pass the output text through the postprocessing filter defined in file. See the chapter 'Filters' (see Filters) for a description of the format of file. Several filters can be applied in sequence using as many '--filter' and '--user-filter' options as needed. The filters are applied in the order they appear on the command line.

'-f'

'--force'

Force overwrite of output files.

'-F name'

'--format=name'

Select the output format. The valid names are 'byte' and 'utf8'.
If no output format is specified, 'byte' (8 bit) is assumed.

'-i'

'--invert'

Invert image levels (white on black).

'-l'

'--layout'

Enable page layout analysis. Ocrad is able to separate blocks of text of arbitrary shape as long as they are clearly delimited by white space.

'-o file'

'--output=file'

Place the output into file instead of into the standard output.

'-q'

'--quiet'

Quiet operation.

'-s value'

'--scale=value'

Scale up the input image by value before layout analysis and recognition. If value is negative, the input image is scaled down by -value.

'-t name'

'--transform=name'

Perform given transformation (rotation or mirroring) on the input image before scaling, layout analysis and recognition. Rotations are made counter-clockwise.
Try '--transform=help' for a list of valid transformation names.

'-T value'

'--threshold=value'

Set binarization threshold for pgm or ppm files or for '--scale' option (only for scaled down images). value should be a rational number between 0 and 1, and may be given as a percentage (50%), a fraction (1/2), or a decimal value (0.5). Image values greater than threshold are converted to white. The default value is 0.5.

'-u left,top,width,height'

'--cut=left,top,width,height'

Cut the input image by the rectangle defined by left, top, width and height. Values may be relative to the image size (-1.0 <= value <= +1.0), or absolute (abs( value ) > 1). Negative values of left, top are relative to the right-bottom corner of the image. Values of width and height must be positive. Absolute and relative values can be mixed. For example 'ocrad --cut 700,960,1,1' will extract from '700,960' to the right-bottom corner of the image.
The cutting is performed before any other transformation (rotation or mirroring) on the input image, and before scaling, layout analysis and recognition.

'-v'

'--verbose'

Verbose mode.

'-x file'

'--export=file'

Write (export) OCR results file to file (see OCR results file). '-x -' writes to stdout, overriding text output except if output has been also redirected with the '-o' option.

Exit status: 0 for a normal exit, 1 for environmental problems (file not found, invalid flags, I/O errors, etc), 2 to indicate a corrupt or invalid input file, 3 for an internal consistency error (eg, bug) which caused ocrad to panic.

4 Postprocessing the produced text

Filters replace some characters in the text output with different characters and remove some other characters from the output. For example, when recognizing a text that is known to contain just numbers, any character recognized as a 'Z' will probably be a '2'.

Filters don't enable the recognition of characters, just filter them from the output. Use '--charset' to enable the recognition of a character set different from the default ISO-8859-15.

Ocrad provides both built-in filters and user-defined filters.

4.1 User-defined filters

The format of a user-defined filter file (see --user-filter) is very simple. Each line contains either a character conversion or a word that specifies the default behaviour for unlisted characters.

A character conversion is a comma-separated list of quoted characters ('c'), character sets ([0-9A-Z]), character codes (U0063), or character ranges (U0000 - UFFFF), and an optional conversion (an equal sign (=) followed by a quoted character or a character code). The characters in the list are converted to the character in the conversion. If no conversion is specified, the character is left unmodified (converted to itself).

The default behaviour is to discard unlisted characters, i.e. those characters not appearing in the file, either by themselves or included in a set or range. If a line containing just the word 'leave' is found in the file, unlisted characters are left unmodified. If the word is 'mark', unlisted characters are marked as unrecognized.

The destination character of a conversion is considered as listed by default. Every character may be listed more than once, even as part of different conversions. The last conversion affecting a given character is the one that is performed.

Character sets and quoted characters may contain escape sequences.

The character '#' at begin of line or after whitespace starts a comment that extends to the end of the line.

Ranges of characters may be specified in character sets by writing the starting and ending characters with a '-' between them. Thus, '[A-Z]' matches any ASCII uppercase letter. '-' may be specified by placing it first or last. ']' may be specified by placing it first. If the first character after the left bracket is '^', it indicates a "complemented set", which matches any character except the ones between the brackets.

Literals (quoted characters and character sets) are decoded as ISO-8859-15. Character codes are decoded as UCS2. Thus, a 'latin capital letter y with diaeresis' is specified in a set as '[\xBE]', but its code is 'U0178'.

Spaces and control characters are unaffected by filters, except that leadind, trailing, and duplicate spaces produced by the removal of other characters will be themselves removed.

Here is an example user-defined filter file equivalent to the built-in filter 'numbers':

     leave                      # remove this line to get 'numbers_only'
     'D', 'O', 'Q', 'o' = '0'
     'I', 'L', 'l', '|' = '1'
     'Z', 'z'           = '2'
                          '3'
     'A', 'q'           = '4'
     'S', 's'           = '5'
     'G', 'b', U00F3    = '6'   # latin small letter o with acute
     'J', 'T'           = '7'
     '&', 'B'           = '8'
     'g'                = '9'

4.2 Built-in filters

Ocrad provides the following built-in filters (see --filter):

'--filter=letters'

Forces every character that resembles a letter to be recognized as a letter. Other characters will be output without change.

'--filter=letters_only'

Same as '--filter=letters', but other characters will be discarded.

'--filter=numbers'

Forces every character that resembles a number to be recognized as a number. Other characters will be output without change.

'--filter=numbers_only'

Same as '--filter=numbers' but other characters will be discarded.

'--filter=same_height'

Discards any character (or noise) whose height differs in more than 10 percent from the median height of the characters in the line.

'--filter=text_block'

Discards any character (or noise) outside of a rectangular block of text lines.

'--filter=upper_num'

Forces every character that resembles a uppercase letter or a number to be recognized as such. Other characters will be output without change.

'--filter=upper_num_mark'

Same as '--filter=upper_num', but other characters will be marked as unrecognized.

'--filter=upper_num_only'

Same as '--filter=upper_num', but other characters will be discarded.

5 Library version

The application should compare OCRAD_version and OCRAD_version_string for consistency. If the first character differs, the library code actually used may be incompatible with the 'ocradlib.h' header file used by the application.

     if( OCRAD_version()[0] != OCRAD_version_string[0] )
       error( "bad library version" );

6 Library functions

These are the OCRAD library functions. In case of error, all of them return -1 or a null pointer, except 'OCRAD_open' whose return value must be verified by calling 'OCRAD_get_errno' before using it.

7 Library error codes

Most library functions return -1 or a null pointer to indicate that they have failed. But this return value only tells you that an error has occurred. To find out what kind of error it was, you need to verify the error code by calling 'OCRAD_get_errno'.

Library functions do not change the value returned by 'OCRAD_get_errno' when they succeed; thus, the value returned by 'OCRAD_get_errno' after a successful call is not necessarily OCRAD_ok, and you should not use 'OCRAD_get_errno' to determine whether a call failed. If the call failed, then you can examine 'OCRAD_get_errno'.

The error codes are defined in the header file 'ocradlib.h'.

8 Image format conversion

There are a lot of image formats, but ocrad is able to decode only three of them; pbm, pgm and ppm. In this chapter you will find command examples and advice about how to convert image files to a format that ocrad can manage.

'.png'

Portable Network Graphics file. Use the command pngtopnm filename.png | ocrad.
In some cases, like the ocrad.png icon, you have to invert the image with the '-i' option: pngtopnm filename.png | ocrad -i.

'.ps'

'.pdf'

Postscript or Portable Document Format file. Use the command gs -sPAPERSIZE=a4 -sDEVICE=pnmraw -r300 -dNOPAUSE -dBATCH -sOutputFile=- -q filename.ps | ocrad.
You may also use the command pstopnm -stdout -dpi=300 -pgm filename.ps | ocrad,
but it seems not to work with pdf files. Also old versions of pstopnm don't recognize the '-dpi' option and produce an image too small for OCR.

'.tiff'

TIFF file. Use the command
tifftopnm filename.tiff | ocrad.

'.jpg'

JPEG file. Use the command djpeg -greyscale -pnm filename.jpg | ocrad.
JPEG is a lossy format and is in general not recommended for text images.

'.pnm.gz'

Pnm file compressed with gzip. Use the command gzip -cd filename.pnm.gz | ocrad

'.pnm.lz'

Pnm file compressed with lzip. Use the command lzip -cd filename.pnm.lz | ocrad

9 Algorithm

Ocrad is mainly a research project. Many of the algorithms ocrad uses are ad hoc, and will change in successive releases as I myself gain understanding about OCR issues.

The overall working of ocrad may be described as follows:
1) Read the image.
2) Optionally, perform some transformations (cut, rotate, scale, etc).
3) Optionally, perform layout detection.
4) Remove frames and pictures.
5) Detect characters and group them in lines.
6) Recognize characters (very ad hoc; one algorithm per character).
7) Correct some ambiguities (transform l.OOO into 1.000, etc).
8) Optionally, apply one or more filters to the text.
9) Output text result.

As there is no such thing as a free lunch, this approach has some drawbacks. It makes ocrad very sensitive to character defects, and makes difficult to modify ocrad to recognize new characters.

10 OCR results file

Calling ocrad with option '-x' produces an OCR results file (ORF), that is, a parsable file containing the OCR results. The ORF format is as follows:

• Any line beginning with '#' is a comment line.
• The first non-comment line has the form 'source file filename', where filename is the name of the file being processed ('-' for stdin). This is the only line guaranteed to exist for every input file read without errors. If the file, or any block or line, has no text, the corresponding part in the ORF file will be missing.
• The second non-comment line has the form 'total text blocks n', where n is the total number of text blocks in the source image.

For each text block in the source image, the following data follows:

• A line in the form 'text block i x y w h'. Where i is the block number and x y w h are the block position and size as described below for character boxes.
• A line in the form 'lines n'. Where n is the number of lines in this block.

For each line in every text block, the following data follows:

• A line in the form 'line i chars n height h', where i is the line number, n is the number of characters in this line, and h is the mean height of the characters in this line (in pixels).
• N lines (one for every character) in the form 'x y w h; g[, 'c'v]...', where:
  x is the left border (x-coordinate) of the char bounding box in the source image (in pixels).
  y is the top border (y-coordinate).
  w is the width of the bounding box.
  h is the height of the bounding box.
  g is the number of different recognition guesses for this character.
  The result characters follow after the number of guesses in the form of a comma-separated list of pairs. Every pair is formed by the actual recognised char c enclosed in single quotes, followed by the confidence value v, without space between them. The higher the value of confidence, the more confident is the result.

Running ./ocrad -x test.orf testsuite/test.pbm in the source directory will give you an example ORF file.

11 Reporting bugs

There are probably bugs in ocrad. There are certainly errors and omissions in this manual. If you report them, they will get fixed. If you don't, no one will ever know about them and they will remain unfixed for all eternity, if not longer.

If you find a bug in GNU Ocrad, please send electronic mail to bug-ocrad@gnu.org. Include the version number, which you can find by running ocrad --version.

Concept index

• algorithm: Algorithm
• bugs: Problems
• filters: Filters
• getting help: Problems
• image format conversion: Image format conversion
• input charsets: Character sets
• introduction: Introduction
• invoking: Invoking ocrad
• library error codes: Library error codes
• library functions: Library functions
• library version: Library version
• OCR results file: OCR results file
• options: Invoking ocrad
• output format: Character sets
• usage: Invoking ocrad
• version: Invoking ocrad