Skip to content
PathcoreFlow User Manual

Data Ingestion


There are several ways to upload images into the Repository. Depending on the desired workflow and volume of data to be ingested, use one of the following methods:

  • Upload files manually in a browser
  • Upload in bulk from a PC using the Upload Client software
  • Configure the image indexing service by contacting support
  • Upload through the API for customized workflows

Manual Uploads

Files can be manually uploaded into a folder using drag and drop or through a file selection dialog. Multiple images can be selected for upload using either method. The status of uploaded files will be indicated through a progress notification at the top of the screen as well as a progress bar on each new item in the Folder Listing.

To upload file(s) using drag and drop

  1. Navigate to any folder in the Repository (click on the Folder Repository button from the Navigation Menu then navigate to a folder)

  2. Drag one or more files onto the folder listing area (until the “Drop files to upload” notification appears) then drop the files

  3. To check upload status, click on the Details button from the upload progress notification at the top of the screen

To upload file(s) using a file selection dialog

  1. Navigate to any folder in the Repository (click on the Folder Repository button from the Navigation Menu then navigate to a folder)

  2. Click on the Actions button at the top right of the item list

  3. In the popup menu, click Upload

  4. Click in the blue box to activate the file selection dialog or simply drag files into the blue box; repeat as required for other files

  5. Click on the Start Upload button to start upload

  6. Click on the X button to minimize the File Uploads dialog

  7. To check upload status, click on the Details button from the upload progress notification at the top of the screen

To check the status of recently uploaded files

  1. Look for the upload progress notification at the top of the screen

  2. Click on the Details button on the notification

To edit metadata for recently uploaded files

  1. Look for the upload progress notification at the top of the screen and, if it has not been dismissed, click on the Details button. If the notification has been dismissed, this method is not available and file metadata can be edited individually through the Repository listing

  2. Click on the Edit Metadata button for any of the files listed

  3. In the Add fields box look for the fields to be added or type a few letters to find the desired field; repeat as required for other fields

  4. Edit the value in the provided field

  5. Click on the Save Changes button to commit the field(s), or click on the Cancel button to discard changes

  6. Click on the X button in the top right hand corner of the File Uploads dialog

To locate recently uploaded files

  1. Look for the upload progress notification at the top of the screen and, if it has not been dismissed, click on the Details button

  2. Click on the Folder button below any of the recently uploaded files

  3. Click on the X button in the top right hand corner of the File Uploads dialog

To clear the completed list of recently uploaded files

  1. Look for the upload progress notification at the top of the screen and, if it has not been dismissed, click on the Details button

  2. Click on the Clear Completed button

To clear the upload progress history and remove it from view

  1. Look for the upload progress notification at the top of the screen

  2. Click on the Dismiss button on the notification

Refreshing the page while uploads are in progress will terminate in progress uploads and clear the list of recently uploaded files.

Multi-File Images

To upload multi-file images such as MRXS, NDPIS, AFI and VSI via the browser, the set of files and folders related to the image must be uploaded as a single archive file (e.g. a zip file). Do not include additional files or folders in the archive file or change the relative path of files in the archive (i.e. ensure the main file is at the root of the archive and there should be one image per archive).

The following archive formats are supported:

  • tar: Standard tarball archive format
  • tar.gz: gzip compressed tarball
  • ZIP: Created using the DEFLATE or DEFLATE64 algorithm. Microsoft Windows and macOS create these types of archives when a user sends files to a "compressed folder"

The following multi-file image formats are supported, with notes on how to properly organize the archive file contents.

3DHISTECH MRXS

An MRXS image consists of a base .mrxs image, and a correspondingly named folder which contains:

  • Slidedat.ini: Metadata for the entire image
  • Index.dat: Index describing how the data files are combined into a WSI
  • DataXXXX.dat: One or more image data files

Example: An archive file named "slide_001A.zip" containing:

slide_001A.mrxs
slide_001A/
    Data0000.dat
    Data0001.dat
    ...
    Data0021.dat
    Data0022.dat
    Index.dat
    Slidedat.ini
The scanner may not produce the .mrxs file, which is required by PathcoreFlow. CaseViewer will create this file if it does not already exist when it opens the image for viewing.
Aperio AFI

An AFI image consists of:

  • IMAGE_NAME.afi: An XML manifest of the files representing each channel
  • *.svs: One or more images, each representing a channel

Example: An archive file named "273481.zip" containing:

273481_Alexa Fluor 488.svs
273481_Alexa Fluor 594.svs
273481_DAPI.svs
273481.afi
Hamamatsu NDPIS

An NDPIS image consists of:

  • IMAGE_NAME.ndpis: An INI-like manifest of the files representing each channel
  • *.ndpi: One or more images, each representing a channel

Example: An archive file named "Ex_changer.tar.gz" containing:

Ex_changer-DAPI.ndpi
Ex_changer-FITC.ndpi
Ex_changer-TxRed.ndpi
Ex_changer.ndpis
Lunaphore OME-TIFF

An image in the Lunaphore variant of OME-TIFF can have an auxiliary XML file which defines the minimum value, maximum value, gamma value, and pseudocolor for each channel. This XML file needs to match the filename of the OME-TIFF file with ".xml" appended.

Example: An archive file named "Lunaphore 10plex (5 cycles)_Slide 7_Annotation 1.tar" containing:

Lunaphore 10plex (5 cycles)_Slide 7_Annotation 1.ome.tiff
Lunaphore 10plex (5 cycles)_Slide 7_Annotation 1.ome.tiff.xml
Olympus VSI

A VSI image consists of a base .vsi image, and a folder named _IMAGE_NAME_ which can contain a variety of subfolders and files.

Example: An archive file named "Her2.tar" containing:

Her2.vsi
_Her2_/
    stack1/
        frame_t.ets
    stack10001/
        frame_t.ets
    stack10004/
        frame_t.ets
Only one image can be included in an archive file.

Upload Client

Where the volume of images to be uploaded is large, it can be more efficient to upload images with a dedicated tool. The Upload Client is a Windows application that has been designed for automating image uploads. It can be configured to upload images that appear in a “watch” folder. Once the images are uploaded, they are available for review in the Unsorted Uploads section of the application, and from there they can be moved to the appropriate folder in the Repository.

The Upload Client can be configured to detect and upload images that are generated by a whole slide image scanner or it can be used to batch upload images on demand.

Visit the Pathcore Help Center to download the Upload Client and learn more.

Image Indexing Services

[add-on] [on-prem]

The indexing service has been designed for on-premise deployments. The service will find all the images in an attached storage system and reproduce the storage hierarchy in the Repository. The service provides a very efficient mechanism for managing storage in an on-premise deployment.

A side effect of the indexing service is the possibility in some circumstances for duplicate records to be created in the Repository. Images that are moved or copied on the storage system after they have been found by the indexing service will result in duplicate records, as the duplication will be detected by the service as a new file. This is not a huge problem, because both records will function normally (i.e. they will be viewable, no broken links, etc). However, the 2nd record won't have any of the annotations or metadata that may have been associated with the first record.

To reorganize the images in the Repository, we recommend using PathcoreFlow's built-in copy or move functions, which handle image metadata. Note that copy operations in the Repository are shallow, so the image data will not be duplicated on disk.

Custom Built Uploader

Users can develop custom upload workflows using the API, which provides methods for adding files and metadata to the Repository. Contact Pathcore Support to learn more.