Home
Production
Descriptions of OCR processing, TEI transformation, and metadata creation
Entry Files
Procedures for converting single pages into Encyclopedia entries.
Prep and Convert Pages to Entry Files
Before page files can be converted to entry files, we need to do some housekeeping.
Check @-codes
Check @-codes by running a Python script.

Production
Descriptions of OCR processing, TEI transformation, and metadata creation
- Organization
  How to keep hundreds of thousands of files organized.
- Page Files
  Explains the procedures we use to get the best quality OCR of each page.
- Entry Files
  Procedures for converting single pages into Encyclopedia entries.
  - Prep and Convert Pages to Entry Files
    Before page files can be converted to entry files, we need to do some housekeeping.
    - Cleanup Page Files
      Fix the most common OCR errors in the TEI-page files.
    - Review Inventory Comments
      Correct resolvable issues noted in the comments
    - Add Titles to Partial Entries
      Prepare the first entry in the section for inclusion.
    - Check @-codes
      Check @-codes by running a Python script.
    - Convert Pages to Entries
      Use Python to convert an entire section of page files into entry files.
  - Entry-Inventory File
    Document the filenames of every entry in a section using the entry-inventory file.
  - Process Entry Files
    Run cleanup routine on all new entry files
- Master Files
  After creating entry files, we run an automated metadata generation process to add index terms to each entry.

Check @-codes

Check @-codes by running a Python script.

All page files with notes must contain equal numbers of note anchors and note texts codes. In the OCR process, operators mark both using at codes. We check pages for equal numbers by running note_counts.py. The script asks which edition and which section. After checking all files, it outputs a list of differences it discovered, which must be manually corrected, or it prints the message section: @-codes match in all files.