Standardized Workflows (II) Carlos Oscar Sorzano Techn. Director I 2 PC Natl. Center Biotechnology (CSIC)

Interchange Points need an interchange standard Specific proposal in the discussion

Some interchange points CTF estimation of micrographs: List of Micrograph, Voltage, DefocusU, DefocusV, AngleUX, Cs Particle picking: List of Micrograph, micrographXcoor, micrographYcoor Particle extraction: List of images 2D Alignment/Classification: 3D Alignment/Classification: List of image/volume [2D or 3D alignment] [A list of class representatives] [Class representative assignment]

Interchange Proposal Interchange information: – Data: images, volumes and stacks – MetaData: list of … Data structure – Data: Array of real values with X varying faster – MetaData: Table with specific column names Data format – Data: MRC file – MetaData: STAR file with specific block names

Data: Images, volumes and stacks.mrcMRC 2D image.mrcMRC 3D image (can be distinguished from 2D by header).mrcsMRC Stack of 2D images Heymann, J. B.; Chagoyen, M. & Belnap, D. M. J. Structural Biology, 2005, 151, HeaderVal0Val1ValN Val0Val1 x y (0,0) x y

MetaData: Interchange format

MetaData: Components UID: Unique Identifier Micrograph (one or many motifs) Image (motif of interest) Volume CTF Coordinates 2D Alignment 3D Alignment Comment

MetaData: UID, Micrograph, Comment Any entry in the metadata file has a unique entry number listOfMicrographs.3dem: # 3DEM_STAR_1 General file comment # data_block_1 loop_ _UID _micrographLocator _comment 1 InputData/micrograph1.mrc Comment 1 2 InputData/micrograph2.mrc Comment 2 data_block_2 loop_ _UID _micrographLocator 3 InputData/micrographA.mrc 4 InputData/micrographB.mrc

MetaData: CTF Any entry in the metadata file has a unique entry number listOCTFs.3dem: # 3DEM_STAR_1 * # data_block_1 loop_ _UID _Voltage _DefocusU _DefocusV _AngleUX _Cs (in kV) (in μm) (in degrees) (in mm.) X Y U V

MetaData: Images and Volumes Images can be in individual files (mrc) or stacks (mrcs). Volumes are in individual files (mrc). listOfImages.3dem: # 3DEM_STAR_1 * # data_block loop_ _UID _imageLocator 1 image00001.mrc 2 image00002.mrc 3 4 listOfVolumes.3dem: # 3DEM_STAR_1 * # data_block loop_ _UID _volumeLocator 1 volume00001.mrc 2 volume00002.mrc

MetaData: Coordinates Heymann, J. B.; Chagoyen, M. & Belnap, D. M. J. Structural Biology, 2005, 151, x y (0,0) listOfCoordinates.3dem: # 3DEM_STAR_1 * # data_block loop_ _UID _micrographXcoor _micrographYcoor

MetaData: 2D Alignment x y Δx=+5 Δy=-15 Δψ=60° listOfAlignedImages.3dem: # 3DEM_STAR_1 * # data_block loop_ _UID _imageLocator _imageXOff _imageYOff _imagePsiOff

MetaData: 2D Alignment x y listOfAlignedImages.3dem: # 3DEM_STAR_1 * # data_alignedImageList_A loop_ _UID _imageLocator _imageXOff _imageYOff _imagePsiOff _imageFlip Δx=+5 Δy=-15 Δψ=60° flip

MetaData: 2D Alignment x y Δx=+5 Δy=-15 Δψ=60° flip

MetaData: 3D Alignment List of 3D aligned images.3dem: # 3DEM_STAR_1 * # data_block loop_ _UID _imageLocator _homogeneousEulerMatrix I1 mage00001.mrc [a11 a12 a13 Xoff a21 a22 a23 Yoff a31 a32 a33 0] Heymann, J. B.; Chagoyen, M. & Belnap, D. M. J. Structural Biology, 2005, 151, List of 3D aligned volumes.3dem: # 3DEM_STAR_1 * # data_block loop_ _UID _volumeLocator _homogeneousEulerMatrix 1 volume00001.mrc [a11 a12 a13 Xoff a21 a22 a23 Yoff a31 a32 a33 Zoff]

MetaData: Relationships listOfImagesAndDownsampledImages.3dem: # 3DEM_STAR_1 * # data_fullSize_images loop_ _UID _imageLocator 1 2 data_downsampledSize_images loop_ _UID _imageLocator 3 4 data_correspondingDownsampleImage loop_ _UID _UID1 _UID

MetaData: Interchange format There can be any number of blocks within a Star file. Each block can combine any number of components Examples: Micrograph and locations: micrographLocator, micrographXcoor, micrographYcoor Micrograph and CTF: micrographLocator, Voltage, DefocusU, DefocusV, AngleUX, Cs Micrograph, image, location and CTF: micrographLocator, imageLocator, micrographXcoor, micrographYcoor, Voltage, DefocusU, DefocusV, AngleUX, Cs

Some interchange points CTF estimation of micrographs: Block name: micrographCTFs micrographLocator, Voltage, DefocusU, DefocusV, AngleUX, Cs Particle picking: Block name: particlePicking micrographLocator, micrographXcoor, micrographYcoor Particle extraction: Block name: imageList imageLocator 2D Alignment/Classification: 3D Alignment/Classification: A block per class: class_00001, class_00002, … Class blocks: imageLocator [2D or 3D alignment] [A block of class representatives] Blockname: class_representatives imageLocator [Class representative assignment] Blockname: class_representative_assignment UID1, UID2

How to check? Automatic check of results Syntactically and Semantically

Discussion Is this the correct strategy? (interchange points + interchange format) Which interchange points? Which interchange format? How to reach a standard?

Interchange objects CTF determination Micrograph screening Particle picking Particle extraction and screening 2D alignment and classification Initial volume construction 3D Model refinement Micrograph phase correction 3D Model amplitude correction

What do we want to interchange? What is the minimum amount of information needed? How are we going to store it?