TEM Architecture
TEM Architecture

pytemca

AllenInstitute/pytemca

%3 cluster_pyTEM States pyTEM States 7cfdae0a757c48328089f299b5e1657b Preview/Setup State a3d8e92030744553a4b57cc84bd0aa70 ROI Initialization State 7cfdae0a757c48328089f299b5e1657b->a3d8e92030744553a4b57cc84bd0aa70 d314dc769dc34c11a5cd0ee1d05f625b Acquisition State a3d8e92030744553a4b57cc84bd0aa70->d314dc769dc34c11a5cd0ee1d05f625b 3555bf5ac1ba4921b5b53f9681d97c97 Error State a3d8e92030744553a4b57cc84bd0aa70->3555bf5ac1ba4921b5b53f9681d97c97 Error d314dc769dc34c11a5cd0ee1d05f625b->7cfdae0a757c48328089f299b5e1657b Abort d314dc769dc34c11a5cd0ee1d05f625b->a3d8e92030744553a4b57cc84bd0aa70 Repeat d314dc769dc34c11a5cd0ee1d05f625b->3555bf5ac1ba4921b5b53f9681d97c97 Error

Preview State

This state is is the entry point to the system. It is the default state at start up and will also be the state that the system will revert to upon detecting an error that does not crash the system.

Preview State Logic

Upon entering this state the system will continuously acquire images from the XIMEA camera and each image frame will be minimally processed. The images are then JPEG encoded and sent to python which pushes them to a websocket to display on a browser based UI.

%3 cluster_opencv_graph opencv_graph cluster_pyTEMCA pyTEMCA 96ad269f9a8d458eb8aed78254a386ed Acquire Image 5ac78882d38b4fe8985e0982a0004f65 Flat Field Correction and CLAHE (CUDA) 96ad269f9a8d458eb8aed78254a386ed->5ac78882d38b4fe8985e0982a0004f65 710673af01f741f691e0f0bdaea1ee70 Calculate Stats (Min/Max/Mean) 5ac78882d38b4fe8985e0982a0004f65->710673af01f741f691e0f0bdaea1ee70 4f0117a7fb944160bf81c10af3807677 Calculate FFT Focus Score 5ac78882d38b4fe8985e0982a0004f65->4f0117a7fb944160bf81c10af3807677 5b3ad39b6f5a4925aeec9a89b267bd89 JPEG Compress 5ac78882d38b4fe8985e0982a0004f65->5b3ad39b6f5a4925aeec9a89b267bd89 e446d0e46b1244bca47e0509c12a205f Send to Python via pybind11 710673af01f741f691e0f0bdaea1ee70->e446d0e46b1244bca47e0509c12a205f 4f0117a7fb944160bf81c10af3807677->e446d0e46b1244bca47e0509c12a205f 5b3ad39b6f5a4925aeec9a89b267bd89->e446d0e46b1244bca47e0509c12a205f 7988d271d8154991973a9b2de792a653 Encode Image e446d0e46b1244bca47e0509c12a205f->7988d271d8154991973a9b2de792a653 dc1f6d00b23d4b05a42ba4c168297b10 Publish Image 7988d271d8154991973a9b2de792a653->dc1f6d00b23d4b05a42ba4c168297b10 09002e1677e94b11bc32ef5cbafbbc52 Web UI dc1f6d00b23d4b05a42ba4c168297b10->09002e1677e94b11bc32ef5cbafbbc52 3e98e1fac441461395f415537cfb3a15 State Changed? 3e98e1fac441461395f415537cfb3a15->96ad269f9a8d458eb8aed78254a386ed 3e98e1fac441461395f415537cfb3a15->3e98e1fac441461395f415537cfb3a15 Repeat

ROI Initialization State

The ROI Initialization State represents a state for moving to an aperture on a grid tape or stick16.

ROI Initialization State Logic

Upon entering this state it handles the entry logic for the state and initializes tape_ranges if it is None. It tries to fetch barcode values from TEM_db (if using tape); otherwise, it sets default values.

Next the code performs the following:

  1. Initialization and Logging: Clears lists related to aperture critical and error events. Logs the action of changing the aperture.

  2. State Check for Pause: Checks if the current montage state is PAUSE. If yes, it schedules the _change_aperture method to run again after 0.2 seconds and exits the current execution.

  3. Preparatory Actions: Updates beam center results and informs the system that a new montage is starting.

  4. Abort Conditions Check: Checks several conditions to decide whether to abort the operation:
    • If required media information is missing.
    • If environmental conditions are not suitable.
    • If there’s a pending request to abort at the end of the montage.
    • If there is insufficient disk space.
    • If too many errors have accumulated (default is 3).
  5. Handling of Tape and Aperture: Retrieves the next task object (TAO) from the montage controller. If a valid TAO is found:
    • Checks if the destination directory exists or needs to be created.
    • Checks for existing data that might require an abort.
    • Performs adjustments based on the first Region of Interest (ROI) from tao.

  6. Aperture Movement and Correction: Manages movement to the correct grid or aperture based on the tape controller status. Deals with potential barcode errors and adjusts based on confidence levels in the barcode readings.

  7. Lens and Beam Adjustments: Adjusts the beam’s brightness and attempts to find a centroid for the aperture if needed. Conducts a quality check on the beam’s brightness and might abort if the beam mean value is too low. Manages auto-centering of the beam if necessary.

  8. Brightfield Imaging: Handles brightfield imaging conditions and might abort based on the quality of consecutive brightfield images.

  9. Lens Correction Montage: Decides whether a new lens correction montage is necessary based on time intervals and changes in lens conditions.

  10. Finalizing and Error Handling: If no ROIs are found in tao, logs the event but continues processing. If there are still tasks (taos) left in the montage queue, it triggers the start of another attempt. Sets the montage state to NONE if no tasks are left.
%3 cluster_ROI Initialization State ROI Initialization State 00bba2f1568d46a199413115bea7c4ba Enter State 106b3563715a479d9faacb4c7130d021 Initialize tape_ranges 00bba2f1568d46a199413115bea7c4ba->106b3563715a479d9faacb4c7130d021 fbe0652000954d5eaea9f612249496b5 Fetch Barcode from TEM_db 106b3563715a479d9faacb4c7130d021->fbe0652000954d5eaea9f612249496b5 86000e8b16c646e2b5cefe506d364743 Initialization & Logging 106b3563715a479d9faacb4c7130d021->86000e8b16c646e2b5cefe506d364743 f79d213a50154a62b1524e08b03c4852 Check if State is PAUSE 86000e8b16c646e2b5cefe506d364743->f79d213a50154a62b1524e08b03c4852 8d05f3ff2d974e30ab4eb4b893da2796 Retry after 0.2s f79d213a50154a62b1524e08b03c4852->8d05f3ff2d974e30ab4eb4b893da2796 e58a2e0495be4bd7b88a63f2e4fa1692 Preparatory Actions f79d213a50154a62b1524e08b03c4852->e58a2e0495be4bd7b88a63f2e4fa1692 8d05f3ff2d974e30ab4eb4b893da2796->f79d213a50154a62b1524e08b03c4852 585f925e57c047509cd564c6bcc8c2d3 Check Abort Conditions e58a2e0495be4bd7b88a63f2e4fa1692->585f925e57c047509cd564c6bcc8c2d3 e72b7d43e3574fe3aea1a42d748b0541 Handle Tape & Aperture 585f925e57c047509cd564c6bcc8c2d3->e72b7d43e3574fe3aea1a42d748b0541 26e606c9c1e343578bb7c564d1fc1b59 Aperture Movement & Correction e72b7d43e3574fe3aea1a42d748b0541->26e606c9c1e343578bb7c564d1fc1b59 cd6a1a36f1d04402a0613d52b826f819 Lens & Beam Adjustments 26e606c9c1e343578bb7c564d1fc1b59->cd6a1a36f1d04402a0613d52b826f819 1a4eae55f66048c7802d0cc8156e2874 Brightfield Imaging cd6a1a36f1d04402a0613d52b826f819->1a4eae55f66048c7802d0cc8156e2874 e9abd4f3462a4db69c825ccda995d412 Lens Correction Montage 1a4eae55f66048c7802d0cc8156e2874->e9abd4f3462a4db69c825ccda995d412 449f18434eb0499ea82687a9f8130d38 Finalizing & Error Handling e9abd4f3462a4db69c825ccda995d412->449f18434eb0499ea82687a9f8130d38 3094d862b31644e481c1b74096174520 Set State to NONE 449f18434eb0499ea82687a9f8130d38->3094d862b31644e481c1b74096174520

Acquisition State

The Acquisition State represents the overall state of the acquisition loop in the system for imaging and processing data. It manages tasks related to moving the stage, capturing images, processing metadata, and handling various states during the acquisition process.

Acquisition State Logic

Main Acquisition Loop Activation

The acquisition starts if both the main_acquisition_loop_active_event is set and _start_new_aperture() returns True. Initialize variables and prepare for the acquisition sequence within a specific aperture.

ROI Acquisition Process

Iterate over each ROI in roi_list.

For each ROI:

  • Initialize the ROI and calculate the starting position.
  • Move the stage to the initial position and wait for it to stabilize.
  • Enter a nested loop to handle image capture for each tile in the ROI until there are no more positions to move to.
    • Set metadata for the image.
    • Start camera exposure.
    • If there was a previous image, wait for asynchronous processes to complete and possibly write metadata.
    • Wait for the current exposure to complete.
    • Get the next position and move the stage.
    • Handle synchronization and stabilization of the stage.
  • After completing all tiles in the ROI, perform final processing such as writing metadata and resetting the stage to the starting position.

Lens Correction Process

If lens correction is part of the current montage, attempt to correct lens distortions using template matching and solving algorithms. Multiple attempts are made, and if unsuccessful, a critical error event is logged.

Completion and Metadata Handling

Finalize the metadata file and write results to a database. Create a robocopy file if not aborting.

Abort or Switch Event

If an abort signal is received or it’s time to switch to another aperture, the loop is terminated or adjusted accordingly.

Event Scheduling

Schedule callbacks for events like “BACK_TO_SETUP” or “NEXT_APERTURE” based on the conditions met during the loop.

%3 cluster_Outer Infinite Loop Outer Infinite Loop cluster_Main Acquisition Main Acquisition cluster_ROI Acquisition Process ROI Acquisition Process cluster_Per-Tile Per-Tile cluster_Lens Correction Process Lens Correction Process cluster_Completion & Metadata Handling Completion & Metadata Handling cluster_Abort or Switch Event Abort or Switch Event cluster_Event Scheduling Event Scheduling 502e77920f924c46b8e84f5a581bdb3c Start Outer Loop ed05c297eb8a4195bd789303e086b47e Check Activation & Start 502e77920f924c46b8e84f5a581bdb3c->ed05c297eb8a4195bd789303e086b47e b54c8f67cffb446c995dcb1240e26b72 End Outer Loop b54c8f67cffb446c995dcb1240e26b72->502e77920f924c46b8e84f5a581bdb3c a7f5a3b027e54e92b15d3b8f8743d3af Initialize & Prepare Acquisition ed05c297eb8a4195bd789303e086b47e->a7f5a3b027e54e92b15d3b8f8743d3af 16bbf5268ce94ad198252abac902a1c2 For Each ROI a7f5a3b027e54e92b15d3b8f8743d3af->16bbf5268ce94ad198252abac902a1c2 ff6aa29586e94ede8e2f2eaafabc2de7 Initialize ROI & Calculate Position 16bbf5268ce94ad198252abac902a1c2->ff6aa29586e94ede8e2f2eaafabc2de7 247aae0994fe445b83a58abfa8e730ae Move & Stabilize Stage ff6aa29586e94ede8e2f2eaafabc2de7->247aae0994fe445b83a58abfa8e730ae 8516df309fd8440db7145e9c4d1335d7 Set Metadata for Each Tile 247aae0994fe445b83a58abfa8e730ae->8516df309fd8440db7145e9c4d1335d7 899c3843785b4730a3eda18407272416 Finalize ROI 6a0fd4459733491eba5a94b19bf1c2aa Correct Lens Distortions 899c3843785b4730a3eda18407272416->6a0fd4459733491eba5a94b19bf1c2aa b50572cd491e4570bdb6972a624a5f59 Start Camera Exposure 8516df309fd8440db7145e9c4d1335d7->b50572cd491e4570bdb6972a624a5f59 3cc5bc96cc394a088ec16c350aa2c11e Handle Async & Write Metadata b50572cd491e4570bdb6972a624a5f59->3cc5bc96cc394a088ec16c350aa2c11e 641c8a06e3d94aebafcf5b6a6fe15a7a Complete Exposure 3cc5bc96cc394a088ec16c350aa2c11e->641c8a06e3d94aebafcf5b6a6fe15a7a 21160053e2ee4c219c31485137a4db0a Get Next Position 641c8a06e3d94aebafcf5b6a6fe15a7a->21160053e2ee4c219c31485137a4db0a 40a184180a474dbf89cd1c6d281ce206 Stabilize Stage 21160053e2ee4c219c31485137a4db0a->40a184180a474dbf89cd1c6d281ce206 fe71c48645a54e77ab62fed20a55ce3c More Tiles? 40a184180a474dbf89cd1c6d281ce206->fe71c48645a54e77ab62fed20a55ce3c fe71c48645a54e77ab62fed20a55ce3c->899c3843785b4730a3eda18407272416 fe71c48645a54e77ab62fed20a55ce3c->8516df309fd8440db7145e9c4d1335d7 f58570813e99451088af5d9ca9f0c7e0 Finalize Metadata 6a0fd4459733491eba5a94b19bf1c2aa->f58570813e99451088af5d9ca9f0c7e0 c16c8b819884436abadaa4e9f4d89b4e Create Robocopy File f58570813e99451088af5d9ca9f0c7e0->c16c8b819884436abadaa4e9f4d89b4e 1c020954e18949ea9735ed682ab38aaf Check for Abort or Switch c16c8b819884436abadaa4e9f4d89b4e->1c020954e18949ea9735ed682ab38aaf cb4fc963ee914bf382d0f0f249f1dd24 Handle Abort 1c020954e18949ea9735ed682ab38aaf->cb4fc963ee914bf382d0f0f249f1dd24 05ec74782a624c9889ae8cee51ac3e6a Schedule Events cb4fc963ee914bf382d0f0f249f1dd24->05ec74782a624c9889ae8cee51ac3e6a 05ec74782a624c9889ae8cee51ac3e6a->b54c8f67cffb446c995dcb1240e26b72

Error State