Home
Production
Descriptions of OCR processing, TEI transformation, and metadata creation
Organization
How to keep hundreds of thousands of files organized.
Repositories
A guide to the different repositories used to store ocr-project data.
GitHub
Used for data files.
outputs Repository
Explains the content of the outputs repository.
code Folder
The central repository for program code and transformation scripts.

Production
Descriptions of OCR processing, TEI transformation, and metadata creation
- Organization
  How to keep hundreds of thousands of files organized.
  - Edition-Section System
    File organization depends on two basic folder types
  - Folder names
    As the OCR workflow passes through its various stages, production moves into specific folders for each stage. Their names and contents are given below:
  - Repositories
    A guide to the different repositories used to store ocr-project data.
    - Google Drive
      Used for image files.
    - GitHub
      Used for data files.
      - outputs Repository
        Explains the content of the outputs repository.
        autoindex folder
        Collects all materials needed for indexing entry files.
        code Folder
        The central repository for program code and transformation scripts.
        Python Folder
        Python script files.
        digital-editions Folder
        Storage area for editions generated from the master files.
        entry Folder
        Contains the TEI-encoded data after the page files are converted to entry files.
        master Folder
        Contains the master files for creating digital editions.
        metadata Folder
        A collection of files containing metadata for each entry page.
        page Folder
        Pages are individual printed pages in the Encyclopedia.
        records Folder
        A collection of spreadsheets and other documents recording details of the production and analytical work.
    - archive Repository
      Long-term storage of image files
  - Setting Up the Repositories
    Create local copies of the remote repositories
- Page Files
  Explains the procedures we use to get the best quality OCR of each page.
- Entry Files
  Procedures for converting single pages into Encyclopedia entries.
- Master Files
  After creating entry files, we run an automated metadata generation process to add index terms to each entry.

code Folder

The central repository for program code and transformation scripts.

All programming code and stylesheets are stored in the outputs/code folder on GitHub.

Primary Folders

dp: Material kindly shared with us from Distributed Proofreading and Distributed Proofreading Canada.
python: Python scripts for creating entry files from page files and other data processing tasks.
regex: A collection of useful regex scripts for manipulating textual data in our files.
scenarios: Oxygen XML Editor stores certain parameters for each xml-project in .scenario files.
xslt: XSLT scripts are used to transform files to and from the TEI format. They are also used to modify large volumes of XML data.