# Technical background: how the analyses are built This chapter is a map of the source code. For every page of the web app it lists the procedures and modules that produce the result, so that you know where to look when an analysis has to be changed or fixed. All paths are relative to `src/tomotree/`. ## Architecture in three layers Every analysis is split into the same three layers: 1. **Core library (no Streamlit)** – `classes/` (the `Tomo` object and its geometry), `analysis.py`, `physics/`, `resistograph.py`, `scans/`, `util.py`, `transform_image.py`, `dataset.py`. The computation lives here and can be used from Jupyter. 2. **Page module (Streamlit)** – `streamlit/tomograph/`, `streamlit/resistograph/`, `streamlit/scans/`, `streamlit/bg.py`, ... It reads widget values, calls the core library, draws the figures and tables. 3. **App shell** – `streamlit_app/app.py` (login, data directory, menu) and `streamlit_app/page_handlers.py`, which maps menu labels to page modules. Rule of thumb: a **wrong number** is fixed in layer 1, a **wrong widget, label or layout** in layer 2, a **missing or misrouted menu item** in layer 3. ### Request flow ```text streamlit_app/app.py: app() ├─ get_target_directory() choose the dataset directory ├─ streamlit_app/read_config.py: process_config_file() │ └─ dataset.py: read_config(), select_layer(), parse_config() │ → config (dict), data (dict of DataFrames: nodes, times, speeds, ...) ├─ streamlit_app/scan_geometry.py: select_geometry() │ └─ scan_geometry.py: use_scan_geometry() optional geometry from a 3D scan ├─ build_menu_selection() menu defined in streamlit/menu.py └─ page_handlers.PAGES[selected](...) dispatch to the page module (unknown label → streamlit/premium.py: render_premium_page()) ``` ### Adding or changing a page * Menu label and help text: `streamlit/menu.py` (`menu_data`, `help`). * Label → handler: `streamlit_app/page_handlers.py` (`PAGES`). Each handler imports the page module lazily and passes it `data`, `target_dir`, `widgets`, `cfg`, ... * Shared widgets (sliders, radio buttons, toggles) are defined in `streamlit/widgets/__init__.py` and created in `app()` (`setup_sliders`, `setup_radio_buttons`, `setup_toggle`). * The "Reproduce in Jupyter" block at the end of each page is produced by `streamlit/snippet.py` (`show_snippet`, `dataset_lines`, `tomo_lines`, `call`). When you change the call of a core function on a page, change the snippet lines next to it as well, otherwise the Jupyter code no longer reproduces the page. ### The tomogram is shared between pages The tomogram pages store the last `Tomo` object in `st.session_state['tomo']` and the code creating it in `st.session_state['tomo_snippet']`. All postprocessing and resistograph pages work with this object and first call `streamlit_app/common.py: is_tomogram_available()`. Hence a change in the reconstruction affects all pages under *Postprocessing* and *Res & Tomo*. ### Data loading | Task | Where | |---|---| | Directory layout (`tomogram/`, `backgrounds/`, `photos/`, `states/`, `resistograph/`, `scans/`) | `data_layout.py` (`data_dir`, `write_dir`) | | `config.yaml` → nodes, times, speeds, units, layers | `dataset.py` (`parse_config`, `load_dataset`) | | Check that a dataset has the inputs a page needs (sensor positions, times/speeds, resistograph, scans, …) | `dataset.missing_inputs`, `data_layout.has_files`; per-page list `REQUIRES` in `streamlit_app/page_handlers.py`, checked in `app()` before the page is drawn | | YAML reading/writing preserving formatting | `yaml_utils.py` | | Import of tomograph files (Fakopp `.f3d`, PiCUS `.pit`, Arbotom `.abt`) | `device_files.py` (`detect`, `read`, `combine_directions`, `create_yaml`, `config_text`, `find_files`), Fakopp parsing in `f3d_to_yaml.py`; imported into `config.yaml` by `importers.import_source` (`pit_to_yaml.py` is a compatibility wrapper) | | Tomograph file → `data` for `analysis.reconstruct`, without `config.yaml` (Jupyter) | `dataset.read_device_file(path, layer=None)` | | Sensor geometry taken from a 3D scan | `scan_geometry.py` | ## The core: `Tomo` object `classes/tomo.py: Tomo` is assembled from mixins: | File | Mixin | Content | |---|---|---| | `classes/tomo.py` | `Tomo` | constructor: nodes, rays, grid, boundary (spline/polygon/scan outline), bark, `include_only` | | `classes/_tomo_reconstruction.py` | `_TomoReconstructionMixin` | `fix_velocity_EBSI` (anisotropy), `do_EBSI_method`, `do_ERSM_method`, `do_SIRT_method` | | `classes/_tomo_cells.py` | `_TomoCellsMixin` | cells inside the domain (`compute_inside_mask`), `cells_velocity_statistics`, `limit_cell_plot_velocities` | | `classes/_tomo_analysis.py` | `_TomoAnalysisMixin` | `estimate_sound_wood_speed`, `ray_time_comparison`, `check_triangle_inequality`, `optimize_back_tomogram` | | `classes/_tomo_plot.py` | `_TomoPlotMixin` | `plot` (matplotlib), `plot_plotly`, `plot_bokeh`, `plot_angular_dependence`, `plot_sinogram` | Geometry objects: `classes/node.py` (`Node`), `classes/ray.py` (`Ray`, `Segment`), `classes/cell.py` (`Cell`), `classes/grid.py` (`Grid`, including `set_cell_status_from_threshold` and the grid plot). Numerical helpers: `classes/math.py` (ellipse tests, EBSI weights, colours, centroid), `classes/lib.py` (point-in-polygon, numba), `classes/__init__.py` (angles, cell boundary points). The colour map is set globally in `colorbar.py`. The algorithms themselves are described in {doc}`technical_reconstruction_algorithms`. ## Tomograph ### Rays method, EBSI method, ERSM method * Handler: `page_handlers._page_tomogram` * Page: `streamlit/tomograph/tomogram.py: show_tomogram()` * Procedure: 1. widget values → reconstruction parameters (grid `nx`, `ny`, anisotropy, EBSI power, ERSM segmentation, excluded sensors, alignment, bark); bark per node in `set_bark_thickness_per_node()`; 2. `do_tomo()` → cached `analysis.reconstruct()`, which builds `tt.Tomo(...)`, calls `fix_velocity_EBSI()` and then `do_EBSI_method()` or `do_ERSM_method()` (Rays: `do_Rays_method()`, the ray means, thin or fat rays); optional rescaling `Tomo.optimize_back_tomogram()`; the parameters are stored in `t.info['params']` and shown as badges above the tomogram (`streamlit/tomograph/badges.py`); 3. sound wood speed: `speed_calibration.find_sound_speed_estimate()` → `analysis.sound_wood_speed()` → `Tomo.estimate_sound_wood_speed()`; 4. colour scale: `analysis.color_limits()`, `analysis.reference_speed()`, plotted quantity (velocity, velocity², Edyn): `analysis.plot_function()`; 5. drawing: `Tomo.plot()` / `plot_plotly()` / `plot_bokeh()`, background photo `streamlit/bg.py: add_background()`, legend `streamlit/legends.py`; 6. statistics and ray check: `tomogram_statistics()` → `Tomo.cells_velocity_statistics()`, `Tomo.ray_time_comparison()`. * Ray selection: dialog `select_rays()` with the clickable Bokeh plot `streamlit/tomograph/ray_selector.py`; automatic suggestions from `ray_geometry.py` (`stem_outline`, `rays_outside_outline`, `rays_direction_mismatch`, `sensors_in_pairs`). The list `st.session_state['disabled_rays']` (pairs numbered from 1) goes to `analysis.reconstruct(disabled_rays=...)`, which sets both directions of each pair to NaN. * Core modules: `analysis.py`, `classes/*`, `geometry.py` (pith for alignment), `util.py: show_grid`. * `SIRT method`: `analysis.reconstruct(method='SIRT', sirt_...)` calls `do_SIRT_method()` and stores its report in `t.sirt_report`. The page reads the parameters from the `sirt_*` sliders (`tomogram.sirt_params()`; the ray width is in % of the diameter in the GUI and a fraction in `reconstruct`, speed bounds 0 mean automatic). The SIRT keys enter `tomo_params` (and the snippet) only on the SIRT page. Post-processing `rescale` is not offered for SIRT. * `FBP method` (back projection): `analysis.reconstruct(method='FBP', fbp_...)` calls `do_FBP_method()` and stores its report in `t.fbp_report`; widgets `fbp_filtered`, `fbp_damping`, `fbp_ray_width` (`tomogram.fbp_params()`). SIRT and FBP share `_ray_cell_system()` (segments, thin/fat ray matrix) and `_set_cell_slowness()`. Principles: `docs/backprojection.md`. * Post-processing of EBSI/ERSM (sidebar "Post-processing settings"): `rescale` (`optimize_back_tomogram`, linear map of all cells; times = integral of the slowness) and `fbp_correction` / `fbp_correction_damping` (`correct_with_FBP`: the EBSI/ERSM result is the initial model of one FBP step; rays and segments are kept). Report in `t.fbp_correction_report`; the keys enter `tomo_params` only when the correction is on. ### Speed calibration * Handler: `_page_speed_calibration` * Page: `streamlit/tomograph/speed_calibration.py: speed_calibration_app()` * Procedure: choice of tangential rays (`Tomo.tangential_rays`, without disabled rays) and of mean or median → `analysis.sound_wood_speed()` → `Tomo.estimate_sound_wood_speed(set_status=True)` (the used cells get status 1) → `Tomo.plot(status=True)`. * The result is kept in `st.session_state['sound_wood_speed_']` and used by the colour scale and Binary Tomo. ## Postprocessing ### Binary Tomo * Handler: `_page_binary_tomo` * Page: `streamlit/tomograph/binary.py: binary_tomo()`, threshold widgets in `streamlit/tomograph/binary_threshold.py` * Procedure: 1. threshold: `analysis.binary_threshold()` (from colour scale, % of reference, percentile or value); 2. `analysis.binary_section()`: `Grid.set_cell_status_from_threshold()`, areas, centroids, moment of inertia and section modulus of sound and full section: `physics.inertia_2d`, `physics.section_modulus`, `physics.polygon_area`, full outline `analysis.tree_shape()` with `physics/section_properties.py: PolygonSection` (tensor method) and `physics/momentum_rotation.py: compute_second_moment` (fallback); 3. wall thickness: `util.measure_walls()`, `util.wall_sensor_numbers()`, `util.find_status_change_along_line()`; 4. drawing `Tomo.plot(status=True)`, comparison with photo `streamlit/image_comparison.py`, `streamlit/bg.py`. ### Photo assessment * Handler: `_page_photo_assessment` * Page: `streamlit/tomograph/photo_assessment.py: render_photo_assessment()`; polygon drawing by the vendored component `streamlit/_vendor/streamlit_polygon_editor` * Procedure: traced polygons + calibration line → `physics/photo_section.py`: `photo_sections()` (scale, rotation, mirroring into metres), `polygon_results()` (area, I, W of full and sound part), comparison with the tomogram `binary_results(t, threshold)`, table `summary()`; wall thicknesses from the sensors marked on the photo `photo_walls()` with `wall_sensor_indices()`. * Photos and their tracings (`.tracing.json`) are stored via `data_layout.data_dir(..., "photo")` (`photos/`). ### Area selection tool * Handler: `_page_area` * Page: `streamlit/tomograph/area.py: measure_areas()` * Procedure: bounds from the text input → `analysis.area_bins()` (interval of each cell, area table, uses `analysis.plot_function()`) → `analysis.plot_area_bins()`. ### Tomogram measurement tool * Handler: `_page_measure` * Page: `streamlit/tomograph/measure.py: measure_tomogram()` * Procedure: `plot_tomogram()` draws `Tomo.plot()` into an image, clicks are converted to coordinates by `pixel_to_coord()` / `process_click()` (all in the page module, no core function). ### Background * Handler: `_page_background` * Page: `streamlit/bg.py: select_image_for_background()` * Procedure: the user clicks sensors in the photo → `analysis.background_alignment()` → `transform_image.fit_similarity_transform()`; the result is written into the `background:` section of `config.yaml` (`streamlit_app/read_config.py: add_to_config_file`). The photo is later drawn under tomograms by `bg.add_background()` → `transform_image.transform_image()`. ### Dividing line * Handler: `_page_split_line` * Page: `streamlit/tomograph/split_line.py: split_line_page()` * Procedure: parameters of the method from the applied tomogram (`pending.applied()`, otherwise `tomo_params_from_state()`) → `split.split_tomogram()` (whole tomogram, two halves from `split.split_halves()` reconstructed by `analysis.reconstruct(include_only=...)`, merged on the grid of the whole tomogram) → sound wood speed and `analysis.color_limits()` of the whole tomogram → `split.split_metrics()` (`crossing_delay` from `split.crossing_rays()` and `analysis.ray_times()`; the line is taken from `merged.info['split_line']` or passed as `line=`). The line is drawn into both tomograms by `split.plot_line()`. Search: `split.scan_split_lines()` over `split.split_lines()`, drawn by `split.plot_scan()`; the result is kept in `st.session_state['_split_scan']`. * Snippet: `split_snippet()` in the page module. * Settings `split_method`, `split_end_a`, `split_end_b`, `split_min_skipped` are registry widgets and can be saved to `config.yaml`. ### Angle plots * Handler: `_page_angle_plots` (the page is written directly in the handler) * Procedure: `streamlit/tomotree.py: plot_angular_dependence()` → `Tomo.plot_angular_dependence()` (matplotlib or plotly), parameters `angle_params()`, `angle_figsize()`. ## Scan tools The 3D scan pages share state and widgets from `streamlit/scans/__init__.py` (`load_scanner_model`, `render_scanner_settings_widgets`, `compute_cross_section`). The core is in `scans/`. ### Sensor position * Handler: `_sensor_position_tool` * Page: `streamlit/scans/scans_sensors.py` → `streamlit/scans/sensor_geometry_page.py: render_sensor_geometry_page()`; 3D picking in `streamlit/scans/sensor_frontend` * Procedure (`scans/sensor_geometry.py`): `validate_record()` → outline `fit_boundary()` (concave hull) or `mesh_boundary()` (mesh intersection) → `validate_boundary()`, `orient_geometry()`, `move_sensors()`, `reverse_numbering()` → export `geometry_archive()`, `scan_geometry_table()`; import into `config.yaml` with `importers.refresh_source()`. Missing surface: `scans/surface_reconstruction.py: reconstruct_surface()`. Red marker detection: `scans/marker_detection.py: detect_markers()`. Saving to the dataset: `scan_geometry.csv` plus `sensor_geometry.json` next to the scan (`saved_geometry()`, `load_saved_geometry()`, `check_scan_plane()`). * The exported table is read by `scan_geometry.py` during data loading. ### Cross section * Handler: `_cross_section_tool` * Page: `streamlit/scans/scans_trunk.py: render_tree_trunk_scanner_page()`, polygon editing `streamlit/scans/polygon_editor.py` * Procedure: `scans/scantree.py`: `load_scan()` / `load_glb_file()` → `TreeTrunkScanner` (`restrict_height_range`, `rotate_point_cloud`, `find_perpendicular_slice`) → `cross_section(vertices, CutSettings)` → `CrossSection` polygon (concave/convex hull, shapely). 3D view: three.js (`streamlit_3d_viewer.show_3d_viewer`), plotly or pyvista; the polygon editor component is in `streamlit/scans/frontend`. * Tomogram or photo on the cut (three.js): `scans/tomogram_placement.py` (`fit_to_sensors()` with a saved sensor geometry, otherwise `place_at_sensor1()`; image `tomogram_image()`), photo: `streamlit/tomograph/photo_on_cut.py` → `scans/photo_placement.py: fit_photo_to_sensors()`. Jupyter version (PyVista): `scans/scantree.py: tomogram_on_scan()`. ### Cross section mechanical analysis * Handler: `_mechanical_analysis_tool` * Page: `streamlit/scans/scans_cross_section.py: render_tree_cross_section_scanner_page()` * Procedure: polygon from *Cross section* (session state) → `physics/section_properties.py: PolygonSection` (area, perimeter, diameter, `sweep()` = I and W for angles −90…90°, `extrema()`), all computed in `PolygonSection` itself (shapely, tensor or rotation method). ### 3D scan tools * Handler: `_scan_tools` * Page: `streamlit/scans/scans_batch_browser.py: render_scan_batch_browser()` * Procedure: listing of `scans/` directory, OBJ → GLB conversion by the external package `streamlit_3d_viewer.converter.convert`, smoothing by `scans/surface_reconstruction.py: reconstruct_surface()` (Poisson reconstruction in a separate process). ## Resistograph All pages create `resistograph.py: Resistograph(directory=target_dir)`, which reads the files from `resistograph/` (`read_resistograph_data`) and preprocesses them (`preprocess_data`, `remove_global_linear_trend`, Savitzky–Golay smoothing). Checks shared by the pages are in `streamlit/resistograph/__init__.py`. ### Resistograph mean & std * Handler: `_page_resistograph_mean_std` * Page: `streamlit/resistograph/mean_std.py: resistograph_draw()` * Procedure: `resistograph.plot_savgol_mean_std()` or `resistograph.plot_rolling_mean_std()`. ### Resistograph * Handler: `_page_resistograph` * Page: `streamlit/resistograph/curves.py: resistograph_curves_app()` * Procedure: plotly chart of `Resistograph.data`, table and CSV download. ### Define drilling paths * Handler: `_page_drilling_paths` * Page: `streamlit/resistograph/drilling_path.py: drilling_path_setup()` * Procedure: manual paths or `resistograph.auto_drilling_paths(res, tomo)` → `set_drilling_paths()` (`DrillingPath` objects). The paths live in `st.session_state['drill_path_data']`; the *Save to config file* button (`save_drilling_paths_to_config()`) writes them as `drill_path_data` into `config.yaml` (current layer). `initialize_drilling_path_session()` loads them from the config when the session has none; it is called here and on *Res & Tomo*. ### Res & Tomo * Handler: `_page_res_tomo` * Page: `streamlit/resistograph/plot.py: plot_resistogram_on_tomogram()` * Procedure: tomogram `Tomo.plot()` + `set_drilling_paths()` + `resistograph.add_resistograph_to_image()` (coloured bars along the paths) and `add_resistograph_graphs()` (curves along the paths). ## Data & Info & Share | Page | Page module | Core | |---|---|---| | View data tool | `streamlit/showdata.py: show_data()` | `analysis.symmetry_check()`, `tomogram.do_tomo()` | | Share | `streamlit/share.py: share_data()` | `streamlit_app/crypto_helpers.py: encrypt_key()` | ## Premium pages Menu items without a handler in `PAGES` are passed to `streamlit/premium.py: render_premium_page()`, which calls `tomotree_premium.render_page()` from the separately installed `tomotree_premium` package.