Image-related record format
*The interpretation of the Advanced part is only available in EN.
Image processing record (IPR)
Image processing record (IPR) file is designed to record the whole-life information of a microscope image from photo-taking to processing. Each staining image (ssDNA/DAPI, IF, H&E) includes six basic groups, "ImageInfo", "QCInfo", "Stitch", "TissueSeg", "CellSeg", and "Register", which are used to store microscopy photo-taking information, image quality control information, image stitching records, tissue segmentation records, cell segmentation records (optional), and registration records.
| Item | Description |
|---|---|
| Attribute | Description |
| IPRVersion | IPR file format version. |
| Dateset | Description |
| Preview | A 2D matrix merges stitched image, tissue segmentation boundary and cell segmentation boundary. |
| /<stainType>/ImageInfo: Group records basic image information. | |
| Attributes | Description |
| AppFileVer | Microscope software version. |
| BackgroundBalance | Background balance. |
| BitDepth | The bit-depth of a camera sensor describes its ability to transform the analog signal coming from the pixel array into a digital signal. |
| Brightness | Relative intensity affecting a person or sensor. |
| ChannelCount | Number of RGB channels. |
| ColorEnhancement | Whether enhanced image color display or not. |
| Contrast | The difference in color and intensity of the depicted object from its background. |
| DeviceSN | Microscope device serial number. |
| DistortionCorrection | Whether fixed distortion or not. |
| ExposureTime | Exposure time in ms. |
| FOVHeight | Height of an individual FOV in pixel. |
| FOVWidth | Width of an individual FOV in pixel. |
| Gain | Amplification applied to the signal by the image sensor. |
| Gamma | The coefficient links between the human eye and the digital camera. |
| GammaShift | Whether adapt the digital image taken with the help of a linearly recording camera to the nonlinear perception of the human eye or not. |
| Illuminance | Intensity of light. |
| Manufacturer | Microscope manufacturer. |
| Model | Microscope model. |
| Overlap | Overlapping pixels between single tiles. |
| PixelSizeX | Size of pixel in x direction. |
| PixelSizeY | Size of pixel in y direction. |
| QCResultFile | Prefix of ImageQC/ImageStudio result file, the unique identifier of the image. |
| ScanChannel | Fluorescence channel. |
| ScanCols | Number of columns scanned. |
| ScanObjective | Magnification power of the scan objective lens. |
| ScanRows | Number of rows scanned. |
| ScanTime | Scan date and time. |
| Sharpness | Degree of clarity of the edge(s) of the image. |
| STOmicsChipFovRow | Stereo-seq Chip mask‘s FOV row. |
| STOmicsChipFovCol | Stereo-seq Chip mask‘s FOV column. |
| RegisterVersion | Image auto-processing & matrix registration software version. |
| StitchedImage | Whether the corresponding image is a panorama image (true) or a set of tiled images (false). |
| STOmicsChipSN | Stereo-seq Chip serial number. |
| WhiteBalance | An adjustment in electronic and film imaging that corrects the color balance of the lighting. |
| DatasetDataType: 1D array | Description |
| RGBScale | RGB color. |
| /<stainType>/QCInfo: Group records the QC information of the image. | |
| Attributes | Description |
| Experimenter | Email of the experimenter who did QC for the image. |
| GoodFOVCount | Number of FOVs that have identified more than 3 track cross points. |
| ImageQCVersion | Image QC version. |
| QCPassFlag | Whether the corresponding image passed QC. |
| RemarkInfo | Any remarks, notes, comments on the image. |
| StainType | Stain type of microscopy image. |
| TotalFOVCount | Total number of FOVs. |
| TrackLineScore | Reference score for evaluating whether the detected track lines can be used as references for image stitching and registering with gene expression matrix. (This score only evaluate whether the program detected track lines on the image, it does not infer the clarity of the lines or the images). |
| TrackLineChannel | Shooting channel of track lines. |
| TrackCrossQCPassFlag | Whether the QC of the track-cross point is passed. |
| ScopeStitchQCScore | Score to assess the stability of microscope overlap (DAPI & mIF only). |
| ScopeStitchQCPassFlag | Whether the QC of microscope stitch is passed (DAPI & mIF only). |
| TemplateValidArea | Proportion of encircled area by the detected points that match the global track line template (within the error range of 5 pixels), to the entire image area. |
| TemplateRecall | Proportion of detected points, which match the global track line template (within the error range of 5 pixels), to the points derived from the chip track line rules. |
| DatasetDataType: 2D array | Description |
| ScopeStitchQCMatrix | Matrix of overlap deviation of each microscope FOV (DAPI & mIF only). |
| CrossPoints/row_col*n | Group of datasets for each FOV that records the track cross point coordinates. (Row and col stand for the FOV row and column index number, and n stands for number of FOVs). Each dataset is a 2D array record, (x, y) coordinates of track cross points in each FOV. |
| /<stainType>/Calibration: Group records the calibration information (<protein>_IF only). | |
| Attributes | Description |
| CalibrationQCPassFlag | Whether the calibration QC of IF image is passed. |
| /<stainType>/Calibration/Scope: | |
| Attributes | Description |
| Confidence | Calibration confidence of microscope-stitched tiled image vs. IF. |
| OffsetX | Horizontal offset. |
| OffsetY | vertical offset. |
| [optional]/<stainType>/Calibration/Stereo: | |
| Attributes | Description |
| Confidence | Calibration confidence of Stereo-stitched image vs. IF. |
| OffsetX | Horizontal offset. |
| OffsetY | Vertical offset. |
| /<stainType>/Stitch: Group records the stitching information. | |
| Attributes | Description |
| StitchingScore | Reference score for stitching. |
| TemplateSource | The reference FOV for deriving the template used for rotating and scaling the microscopic images. |
| WhichStitch | Stitching method for image, including microscope, template and ripple stitching. The default for stitched image is microscope. |
| DatasetDataType: 2D array | Description |
| TemplatePoint | Center coordinates for deriving template lines. |
| TransformTemplate | Coordinates of template points registered with expression matrix. |
| /<stainType>/Stitch/StereoStitch: Group records the image stitching information processed by Stereo program. | |
| Attributes | Description |
| StitchedGlobalHeight | Height of stitched tiled images using Stereo stitching algorithm. Tiled image only. |
| StitchedGlobalWidth | Width of stitched tiled images using Stereo stitching algorithm. Tiled image only. |
| DatasetDataType: 2D array | Description |
| StitchedGlobalLoc | Coordinates for the Stereo stitched tiled image. Tiled image only. |
| /<stainType>/Stitch/ScopeStitch: Group records the image stitching information processed by microscope imaging software. | |
| Attributes | Description |
| GlobalHeight | Height of panorama image. |
| GlobalWidth | Width of panorama image. |
| DatasetDataType: 2D array | Description |
| GlobalLoc | Coordinates for the stitched tiled image (either program stitched or microscope stitched). |
| ScopeJitterDiff | Jitter offset of microscope stitching. |
| ScopeHorizontalJitter | Horizontal jitter offset of microscope stitching. Tiled image only. |
| ScopeVerticalJitter | Vertical jitter offset of microscope stitching. Tiled image only. |
| /<stainType>/Stitch/StitchEval: Group records the evaluation result of stitching. | |
| Attribute | Description |
| MaxDeviation | Maximum stitching deviation. |
| DatasetDataType: 2D array | Description |
| GlobalDeviation | Global stitching deviation matrix. Tiled image only. |
| StitchEvalH | Stitching deviation matrix for the horizontal axes. |
| StitchEvalV | Stitching deviation matrix for the vertical axes. |
| /<stainType>/Register: Group records the information that aligns images with gene expression matrix. | |
| Attributes | Description |
| CounterRot90 | Count of counter-clockwise rotation of 90 degrees. |
| Flip | Whether horizontally flipped or not. |
| MatrixShape | Height and width of the gene expression matrix. |
| OffsetX | Offset between microscope image and gene expression matrix in x-axis. |
| OffsetY | Offset between microscope image and gene expression matrix in y-axis. |
| RegisterScore | Reference score for registration. |
| Rotation | Rotation degree between raw image and deviation template. |
| ScaleX | Scale between raw image and deviation template in horizontal direction. |
| ScaleY | Scale between raw image and deviation template in vertical direction. |
| XStart | Gene expression matrix offset x (GEF geneExp/binN/expression attribute minX). |
| YStart | Gene expression matrix offset y (GEF geneExp/binN/expression attribute minY). |
| ManualRotation | Manual rotation degree of the raw image around the center point. |
| ManualScaleX | Manual scale of the raw image in horizontal direction based on center point. |
| ManualScaleY | Manual scale of the raw image in vertical direction based on center point. |
| DatasetDataType: 2D array | Description |
| MatrixTemplate | List of track cross points derived from gene expression matrix. |
| /<stainType>/TissueSeg: Group records the tissue segmentation information. | |
| Attributes | Description |
| TissueSegScore | Reference score for tissue segmentation. |
| TissueSegShape | Image shape for tissue segmentation mask image. |
| Threshold | The threshold for tissue segmentation in IF images is divided into maximum and minimum values (IF only). |
| DatasetDataType: 2D array | Description |
| TissueMask | Encoded tissue segmentation mask file (after registration with gene expression matrix). |
| /<stainType>/CellSeg: Group records the cell segmentation information. | |
| Attributes | Description |
| CellSegShape | Image shape for cell segmentation mask image. |
| DatasetDataType: 2D array | Description |
| CellMask | Encoded cell segmentation mask file (after registration with gene expression matrix). |
| CellSegTrace | Cell contour attributes, including height, width and area. |
| /StereoResepSwitch: Group stores the state of each module that whether the module needs to be performed. | |
| Attributes | Description |
| stitch | Switch for performing stitching. |
| tissueseg | Switch for performing tissue segmentation. |
| cellseg | Switch for performing cell segmentation. |
| register | Switch for performing registration. |
| /ManualState: Group stores the state of each module that whether the module has been manually processed. | |
| Attributes | Description |
| stitch | Whether manually stitched the tiled images. |
| tissueseg | Whether manually delineated the tissue coverage region. |
| cellseg | Whether manually delineated the cell coverage regions. |
| register | Whether manually aligned microscope image and gene expression matrix. |
| calibration | Whether manually calibrated two images. For example, manually match IF image with DAPI image. |
Recorded image processing (RPI)
The image pyramid model is a multi-resolution hierarchical model that is used to store and display images in different resolutions. For the same field of view, the layer of the image pyramid that is closest to the bottom includes the most detailed information and has the largest scale. register pipeline performs the down-sampling step on the registered image, and the result images are layered to construct a pyramid with the suffix ".rpi". For each resolution layer, the intact registered image is split into 256 pixels x 256 pixels tiles. If the size of a layer is smaller than 256 x 256, the image will then remain intact. The outer layer group of RPI file is defined according to its stain type, generally including ssDNA, DAPI, IF (immunofluorescence image especially for protein, group name as <protein_IF>), and H&E. In each group of stain types, multiple image results, including an image subgroup (registered microscopy image), a TissueMask subgroup (registered mask boundary for the tissue coverage area), and a CellMask subgroup (registered mask boundaries for the cell coverage area, optional), could be saved respectively.
Schematic diagram of the image pyramid:
.png)
Image pyramid displaying
| Item | Description |
|---|---|
| Attributes | Description |
| imgSize | Size of the image. |
| sizex | Max of x. |
| sizey | Max of y. |
| version | RPI file version. |
| x_start | Offset x of the matrix. |
| y_start | Offset y of the matrix. |
| /\ |
|
| Attributes | Description |
| MaxGrayLevel | Maximum gray value. |
| MinGrayLevel | Minimum gray value. |
| GrayLevelElbow | Elbow point of gray value. |
| TrackLayer | Whether track layer. |
| Color | Layer color. |
| Groups | Description |
| binN | Detailled downsampling metadata for each bin size. |