extract_experiment_data_from_video

What it does

General-purpose extractor that turns lab video streams into typed, timestamped measurements (e.g., liquid volume, color/turbidity, cell/colony counts, pipette readouts, instrument displays, gel band intensities) and emits a time-series JSON or CSV table for downstream analysis or ELN attachment.

How it works

• Ingests video in formats like MP4, AVI, MOV, MKV, TIFF stack, OME-TIFF or XR streams via RTSP/WebRTC.
• Allows ROI definitions (named rectangles/polygons) or auto-ROI detection via scene understanding; applies perspective correction for XR footage and temporal alignment across multiple cameras.
• Runs multiple extractors in parallel or sequence, including:
  • Liquid Volume & Level Tracking: detect liquid level per vessel, map to volume, detect events, output fields: timestamp_s, vessel_id, fill_level_px, volume_estimated_uL, delta_volume_uL, event_type.
  • Color & Turbidity Extraction: compute RGB, HSV, Lab* per ROI; optional in-frame calibration to concentration; compute relative_turbidity; outputs: timestamp_s, roi_id, R, G, B, L_star, a_star, b_star, hue_deg, saturation, relative_turbidity, calibrated_concentration.
  • Cell/Colony/Object Counting: detect cells/colonies/objects, output count metrics, confidence, size distributions, and events; outputs fields like timestamp_s, roi_id, object_type, count, mean_area_px2, mean_area_um2, confidence, size_distribution_json.
  • Instrument Display OCR: read numeric/alphanumeric displays with OCR; parse values and units; normalize to SI units; outputs: timestamp_s, instrument_id, display_roi, raw_text, parsed_value, unit, si_value, change_flag.
  • Gel Electrophoresis Quantification: detect gel bands, lanes, migration distances, estimate MW, compute band intensities; outputs: lane_id, band_id, migration_distance_mm, estimated_mw_kda, band_intensity_au, relative_intensity, ladder_r2.
  • Structured Output Schema: writes a unified JSON with video_id, source, extraction_config, timeseries per extractor, events, and warnings, plus a flat CSV export.
• Outputs include time-series and events such as ADDITION, COLOR_TRANSITION, PH_CHANGE, etc., as shown in the example schema.

When to use it

Use when videos capture lab processes where quantitative data must be extracted automatically from imagery, including:

• Colorimetric or turbidity assays over time
• Volume monitoring in vessels
• Pipette readouts and protocol auditing via XR displays
• Instrument display readings in frame
• Cell/colony/spot counting
• Gel band quantification
• Multi-step reaction monitoring and passive ambient logging

What it can touch

• Inputs: video files (MP4, AVI, MOV, MKV, TIFF stack, OME-TIFF) and XR streams (RTSP/WebRTC)
• Outputs: JSON timeseries, flat CSV export
• OCR and image analysis subsystems (e.g., 7-segment/LCD OCR, color/turbidity analysis, ROI-based measurements)

Caveats

• License: MIT
• Notes: relies on a battery of computer-vision extractors; calibration may be required for concentration or OD mapping; auto-ROI can propose targets but user confirmation may be needed; some frames may produce occlusions or glare requiring warnings in output.