Parameter optimization for phase and fluorescence reconstructions (mehta-lab#525)

* Add model-level reconstruct() convenience functions

Each physics model (phase_thick_3d, isotropic_thin_3d,
isotropic_fluorescent_thick_3d, isotropic_fluorescent_thin_3d,
inplane_oriented_thick_pol3d) gets a reconstruct() function
that chains generate_test_phantom + calculate_transfer_function +
apply_transfer_function + apply_inverse_transfer_function.

* Add per-model API layer (waveorder.api)

Each physics model (birefringence, phase, fluorescence,
birefringence_and_phase) gets its own module with Settings,
simulate(), compute_transfer_function(),
apply_inverse_transfer_function(), and reconstruct().

All API functions accept/return xr.DataArray/Dataset.
Shared utilities in _settings.py and _utils.py.

* Refactor CLI to delegate to per-model API modules

CLI settings.py now re-exports from waveorder.api modules.
compute_transfer_function.py and apply_inverse_transfer_function.py
dispatch to birefringence, phase, fluorescence, or
birefringence_and_phase API functions. Removes apply_inverse_models.py.

* Add CLI simulate, view, and download commands

New commands: wo sim, wo view, wo download-examples.
Adds AliasGroup for short aliases (rec, sim, dle, gui).
Updates main.py help formatting. Adds wo entry point.

* Add API and CLI examples

6 API examples (birefringence, birefringence_and_phase, phase_2d,
phase_3d, fluorescence_2d, fluorescence_3d) and matching CLI
shell scripts with YAML config files.

* Update tests for new API architecture

Rewrite API tests to use per-model imports. Update channel names
(BF -> Transmittance, Pol -> Depolarization). Add parametrized
example runner tests for API and CLI examples.

* Update model examples and configs README

Align model example parameters (4-State scheme, yx_pixel_size=0.1,
z_position_list sign convention). Update configs README.

* Update internal callers to use API transfer function settings

Replace BirefringenceTransferFunctionSettings,
PhaseTransferFunctionSettings, FluorescenceTransferFunctionSettings
with direct imports from waveorder.api modules in plugin,
calibration, and test code.

* Simplify API examples to only use the API

Remove model-level imports (torch, waveorder.models, waveorder.api._utils)
and comparison/verification code. Each example now shows only the API:
settings, simulate, compute_transfer_function,
apply_inverse_transfer_function, reconstruct, and print output.

* Simplify configs README

Replace outdated instructions, zenodo links, and TODO with a short note
that configs are auto-generated by test_generate_example_settings.

* Delete download-examples command

Remove download.py, its import/registration/alias in main.py,
and its reference in the view command docstring.

* Move view to end of help, add [v] alias

* Simplify test_examples.py with glob parametrization

Replace explicit file lists with glob("*.py") and glob("*.sh") so new
examples are picked up automatically.

* Consolidate duplicated _make_czyx test helpers into shared fixture

Replace 5 near-identical _make_czyx / _make_czyx_data / _make_czyx_xarray
helpers across test files with a single make_czyx factory fixture in
tests/conftest.py. Also remove duplicate test_position_list_from_shape_scale_offset
from cli tests (superset exists in api tests).

* Documented configs

* Move configs README and delete old configs directory

* Remove expensive contrast limit computation from CLI

* smaller arrays to relive test bottleneck

* Migrate to uv, ruff, and hatch (mehta-lab#523)

* Migrate build system from setuptools to hatchling + uv-dynamic-versioning

* Replace black/isort/autoflake with ruff config

* Apply ruff formatting to codebase

* Migrate CI workflows to uv

* Migrate ReadTheDocs to uv

* Update CONTRIBUTING.md for uv workflow

* Generate and commit uv.lock

* Fix trailing whitespace and missing newlines in non-Python files

* Add formatting commits to git-blame-ignore-revs

* Update install instructions across docs for uv and removed [all] extra

* Update install instructions and keep [all] extra for backward compatibility

[all] is now an alias for [visual] since dev/docs/test deps moved to
[dependency-groups] (PEP 735). Extras ([project.optional-dependencies])
are published to PyPI for end users; dependency groups are local-only
for contributors via uv sync --group <name>.

* Update uv.lock

* Use --only-group for minimal CI/RTD installs, add pre-commit to dev group

* Add waveorder/optim/ module with OptimizableFloat type, losses, logging, and optimizer loop

* Make optics differentiable: smooth pupil, tilted pupil on Ewald sphere, gradient-safe kernels

* Make models differentiable: remove in-place ops, accept tensor params, add pseudo-SVD and tilt

* Add OptimizableFloat and tilt fields to settings, split base vs optimizable settings classes

* Add optimize() to phase and fluorescence API modules

* Add CLI optimization support: OptimizationSettings, optimizable param detection, birefringence guard

* Add tensorboard dependency and update example configs with tilt fields

* Add optimization examples and tests

* add comment for using your own data

* Handle numpy input in generate_pupil for backwards compatibility

* Clean up steepness defaults: use 1e4 consistently, fix docstring

* Make tilt integration test check loss improvement instead of convergence tolerance

* iohub depend on pypi

* Reduce 3D midband_power_loss to scalar for optimizer

* Use optimized config path for reconstruction after optimization

* feat: Added to_device to be target cuda

Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>

* style: formatting

Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>

* Remove z_focus_offset="auto" option and auto-focus CLI code

* Fix 3D optimize loss reduction and parameter name unpacking in phase.py

Reduce per-slice (Z,) loss to scalar before backward pass, matching
fluorescence.py. Use split(".")[-1] for dotted parameter names.

* Revert device/CUDA changes and fix phase.py bugs

* Add numpy-style docstrings to public API functions

* Add 3D optimization integration tests for phase and fluorescence

* Apply missing api-refactor changes: rename validators, remove vector_transfer_function

* Fix generate_pupil callers to pass tensors instead of numpy arrays

* Remove '(or auto)' from z_focus_offset config comments

* bump iohub

* cleaner docs on pupil

* cleaner documentation of z_position_list and na_det

* enable tilt_angle_{zenith,azimuth} for phase_thick_3d

* remove unecessary branching logic

* remove parallel SVD paths...prefer one-shot SVD when available

* synchronize fluorescence and phase optimization loops

* simplify loss handling

* remove redundant _get helper

* clean resolution of optimizable floats

* clean and test optimization cli

* [o] -> (optimizable)

* document pupil steepness choice

---------

Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Co-authored-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>

Author: talonchandler

docs/examples/cli/configs/birefringence-and-phase_3d.yml

@@ -20,10 +20,12 @@ phase:
     yx_pixel_size: 0.1                      # lateral pixel size in micrometers
     z_pixel_size: 0.25                      # axial pixel size in micrometers
     z_padding: 0                            # z slices to pad for axial boundary effects
-    z_focus_offset: 0                       # offset from center slice in slice units
+    z_focus_offset: 0                       # (optimizable) offset from center slice in slice units
     index_of_refraction_media: 1.3          # refractive index of imaging media
-    numerical_aperture_detection: 1.2       # detection objective numerical aperture
-    numerical_aperture_illumination: 0.9    # condenser numerical aperture
+    numerical_aperture_detection: 1.2       # (optimizable) detection objective numerical aperture
+    tilt_angle_zenith: 0.0                  # (optimizable) illumination tilt zenith angle in radians
+    tilt_angle_azimuth: 0.0                 # (optimizable) illumination tilt azimuth angle in radians
+    numerical_aperture_illumination: 0.9    # (optimizable) condenser numerical aperture
     invert_phase_contrast: false            # invert contrast for positive/negative phase
   apply_inverse:
     reconstruction_algorithm: Tikhonov      # 'Tikhonov' or 'TV' regularization

docs/examples/cli/configs/fluorescence_2d.yml

@@ -7,9 +7,11 @@ fluorescence:
     yx_pixel_size: 0.1                      # lateral pixel size in micrometers
     z_pixel_size: 0.25                      # axial pixel size in micrometers
     z_padding: 0                            # z slices to pad for axial boundary effects
-    z_focus_offset: 0                       # offset from center slice in slice units
+    z_focus_offset: 0                       # (optimizable) offset from center slice in slice units
     index_of_refraction_media: 1.3          # refractive index of imaging media
-    numerical_aperture_detection: 1.2       # detection objective numerical aperture
+    numerical_aperture_detection: 1.2       # (optimizable) detection objective numerical aperture
+    tilt_angle_zenith: 0.0                  # (optimizable) illumination tilt zenith angle in radians
+    tilt_angle_azimuth: 0.0                 # (optimizable) illumination tilt azimuth angle in radians
     wavelength_emission: 0.532              # emission wavelength in micrometers
     confocal_pinhole_diameter: null         # confocal pinhole diameter (null = widefield)
   apply_inverse:

docs/examples/cli/configs/fluorescence_3d.yml

@@ -7,9 +7,11 @@ fluorescence:
     yx_pixel_size: 0.1                      # lateral pixel size in micrometers
     z_pixel_size: 0.25                      # axial pixel size in micrometers
     z_padding: 0                            # z slices to pad for axial boundary effects
-    z_focus_offset: 0                       # offset from center slice in slice units
+    z_focus_offset: 0                       # (optimizable) offset from center slice in slice units
     index_of_refraction_media: 1.3          # refractive index of imaging media
-    numerical_aperture_detection: 1.2       # detection objective numerical aperture
+    numerical_aperture_detection: 1.2       # (optimizable) detection objective numerical aperture
+    tilt_angle_zenith: 0.0                  # (optimizable) illumination tilt zenith angle in radians
+    tilt_angle_azimuth: 0.0                 # (optimizable) illumination tilt azimuth angle in radians
     wavelength_emission: 0.532              # emission wavelength in micrometers
     confocal_pinhole_diameter: null         # confocal pinhole diameter (null = widefield)
   apply_inverse:

docs/examples/cli/configs/phase_2d.yml

@@ -8,10 +8,12 @@ phase:
     yx_pixel_size: 0.1                      # lateral pixel size in micrometers
     z_pixel_size: 0.25                      # axial pixel size in micrometers
     z_padding: 0                            # z slices to pad for axial boundary effects
-    z_focus_offset: 0                       # offset from center slice in slice units
+    z_focus_offset: 0                       # (optimizable) offset from center slice in slice units
     index_of_refraction_media: 1.3          # refractive index of imaging media
-    numerical_aperture_detection: 1.2       # detection objective numerical aperture
-    numerical_aperture_illumination: 0.9    # condenser numerical aperture
+    numerical_aperture_detection: 1.2       # (optimizable) detection objective numerical aperture
+    tilt_angle_zenith: 0.0                  # (optimizable) illumination tilt zenith angle in radians
+    tilt_angle_azimuth: 0.0                 # (optimizable) illumination tilt azimuth angle in radians
+    numerical_aperture_illumination: 0.9    # (optimizable) condenser numerical aperture
     invert_phase_contrast: false            # invert contrast for positive/negative phase
   apply_inverse:
     reconstruction_algorithm: Tikhonov      # 'Tikhonov' or 'TV' regularization

docs/examples/cli/configs/phase_3d.yml

@@ -8,10 +8,12 @@ phase:
     yx_pixel_size: 0.1                      # lateral pixel size in micrometers
     z_pixel_size: 0.25                      # axial pixel size in micrometers
     z_padding: 0                            # z slices to pad for axial boundary effects
-    z_focus_offset: 0                       # offset from center slice in slice units
+    z_focus_offset: 0                       # (optimizable) offset from center slice in slice units
     index_of_refraction_media: 1.3          # refractive index of imaging media
-    numerical_aperture_detection: 1.2       # detection objective numerical aperture
-    numerical_aperture_illumination: 0.9    # condenser numerical aperture
+    numerical_aperture_detection: 1.2       # (optimizable) detection objective numerical aperture
+    tilt_angle_zenith: 0.0                  # (optimizable) illumination tilt zenith angle in radians
+    tilt_angle_azimuth: 0.0                 # (optimizable) illumination tilt azimuth angle in radians
+    numerical_aperture_illumination: 0.9    # (optimizable) condenser numerical aperture
     invert_phase_contrast: false            # invert contrast for positive/negative phase
   apply_inverse:
     reconstruction_algorithm: Tikhonov      # 'Tikhonov' or 'TV' regularization

docs/examples/maintenance/PTI_simulation/PTI_Simulation_Forward_2D3D.py

@@ -19,6 +19,7 @@
 
 import matplotlib.pyplot as plt
 import numpy as np
+import torch
 from numpy.fft import fftshift
 from platformdirs import user_data_dir
 
@@ -337,7 +338,7 @@
 # DPC + BF illumination + PolState (sector illumination)
 
 xx, yy, fxx, fyy = util.gen_coordinate((N, M), ps)
-radial_frequencies = np.sqrt(fxx**2 + fyy**2)
+radial_frequencies = torch.sqrt(torch.as_tensor(fxx**2 + fyy**2, dtype=torch.float32))
 
 Pupil_obj = optics.generate_pupil(radial_frequencies, NA_obj / n_media, lambda_illu / n_media).numpy()
 Source_support = optics.generate_pupil(radial_frequencies, NA_illu / n_media, lambda_illu / n_media).numpy()

docs/examples/maintenance/QLIPP_simulation/2D_QLIPP_forward.py

@@ -19,6 +19,7 @@
 
 import matplotlib.pyplot as plt
 import numpy as np
+import torch
 from numpy.fft import fftshift
 from platformdirs import user_data_dir
 
@@ -70,7 +71,7 @@
 # Subsample source pattern for speed
 
 xx, yy, fxx, fyy = util.gen_coordinate((N, M), ps)
-radial_frequencies = np.sqrt(fxx**2 + fyy**2)
+radial_frequencies = torch.sqrt(torch.as_tensor(fxx**2 + fyy**2, dtype=torch.float32))
 Source_cont = optics.generate_pupil(radial_frequencies, NA_illu, lambda_illu).numpy()
 Source_discrete = optics.Source_subsample(Source_cont, lambda_illu * fxx, lambda_illu * fyy, subsampled_NA=0.1)
 plt.figure(figsize=(10, 10))

docs/examples/optimization/optimize_fluorescence_2d.py

@@ -0,0 +1,46 @@
+"""Example: optimize fluorescence reconstruction parameters.
+
+Simulates fluorescence data with a known z_focus_offset,
+then optimizes from a wrong initial guess to recover it.
+"""
+
+import datetime
+
+from waveorder.api import fluorescence
+from waveorder.optim import OptimizableFloat
+
+# Ground truth parameters
+gt_z_offset = 0.6
+
+# Simulate with ground truth
+gt_settings = fluorescence.Settings(
+    transfer_function=fluorescence.TransferFunctionSettings(
+        z_focus_offset=gt_z_offset,
+    )
+)
+phantom, data = fluorescence.simulate(gt_settings, recon_dim=2, zyx_shape=(11, 256, 256))
+
+# Optimize from wrong initial guess
+opt_settings = fluorescence.Settings(
+    transfer_function=fluorescence.TransferFunctionSettings(
+        z_focus_offset=OptimizableFloat(init=0, lr=0.1),
+    )
+)
+
+log_dir = f"./runs/{datetime.datetime.now():%Y%m%d_%H%M%S}"
+
+optimized_settings, recon = fluorescence.optimize(
+    data,
+    settings=opt_settings,
+    num_iterations=50,
+    midband_fractions=(0.01, 0.5),
+    log_dir=log_dir,
+    log_images=True,
+)
+
+s = optimized_settings.transfer_function
+print(f"\n{'Parameter':<20} {'Ground truth':>12} {'Optimized':>12}")
+print(f"{'z_focus_offset':<20} {gt_z_offset:>12.3f} {s.z_focus_offset:>12.3f}")
+
+print("\nTo view optimization logs, run:")
+print("  tensorboard --logdir ./runs")

docs/examples/optimization/optimize_phase_2d.py

@@ -0,0 +1,70 @@
+"""Example: optimize phase reconstruction parameters.
+
+Simulates brightfield phase data with known z_focus_offset and
+tilt illumination, then optimizes from (0, 0, 0) to recover
+the ground truth parameters.
+"""
+
+import datetime
+
+import numpy as np
+
+from waveorder.api import phase
+from waveorder.optim import OptimizableFloat
+
+# To use your own data instead of simulated data, create a CZYX xr.DataArray:
+#
+#   import xarray as xr
+#
+#   zyx_array = np.load("my_data.npy")      # shape (Z, Y, X)
+#   data = xr.DataArray(
+#       zyx_array[None],                    # add C dimension -> (1, Z, Y, X)
+#       dims=("c", "z", "y", "x"),
+#   )
+#
+# Then pass `data` to `phase.optimize(data, ...)` below.
+
+# Ground truth parameters
+gt_z_offset = 0.6
+gt_tilt_zenith = 0.5
+gt_tilt_azimuth = np.pi / 4
+
+# Simulate with ground truth
+gt_settings = phase.Settings(
+    transfer_function=phase.TransferFunctionSettings(
+        z_focus_offset=gt_z_offset,
+        tilt_angle_zenith=gt_tilt_zenith,
+        tilt_angle_azimuth=gt_tilt_azimuth,
+    )
+)
+phantom, data = phase.simulate(gt_settings, recon_dim=2, zyx_shape=(11, 256, 256))
+
+# Optimize from (0, 0, 0) initial guess
+opt_settings = phase.Settings(
+    transfer_function=phase.TransferFunctionSettings(
+        z_focus_offset=OptimizableFloat(init=0, lr=0.1),
+        tilt_angle_zenith=OptimizableFloat(init=0, lr=0.1),
+        tilt_angle_azimuth=OptimizableFloat(init=0, lr=0.1),
+    )
+)
+
+log_dir = f"./runs/{datetime.datetime.now():%Y%m%d_%H%M%S}"
+
+optimized_settings, recon = phase.optimize(
+    data,
+    settings=opt_settings,
+    num_iterations=50,
+    midband_fractions=(0.1, 0.5),
+    log_dir=log_dir,
+    log_images=True,
+)
+
+s = optimized_settings.transfer_function
+print(f"\n{'Parameter':<20} {'Ground truth':>12} {'Optimized':>12}")
+print(f"{'z_focus_offset':<20} {gt_z_offset:>12.3f} {s.z_focus_offset:>12.3f}")
+print(f"{'tilt_angle_zenith':<20} {gt_tilt_zenith:>12.3f} {s.tilt_angle_zenith:>12.3f}")
+print(f"{'tilt_angle_azimuth':<20} {gt_tilt_azimuth:>12.3f} {s.tilt_angle_azimuth:>12.3f}")
+
+print("\nTo view optimization logs and images, run TensorBoard in your terminal:")
+print("  tensorboard --logdir ./runs")
+print("Then open the displayed URL in your browser.")

docs/examples/optimization/phase_2d_optimized.yml

@@ -0,0 +1,30 @@
+input_channel_names: [Brightfield]
+time_indices: all
+reconstruction_dimension: 2
+phase:
+  transfer_function:
+    wavelength_illumination: 0.532        # illumination wavelength in micrometers
+    yx_pixel_size: 0.1                    # lateral pixel size in micrometers
+    z_pixel_size: 0.25                    # axial pixel size in micrometers
+    z_focus_offset:                       # (optimizable) offset from center slice in slice units
+      init: 0
+      lr: 0.01
+    index_of_refraction_media: 1.3        # refractive index of imaging media
+    numerical_aperture_detection: 1.2     # (optimizable) detection objective numerical aperture
+    numerical_aperture_illumination: 0.9  # (optimizable) condenser numerical aperture
+    tilt_angle_zenith:                    # (optimizable) illumination tilt zenith angle in radians
+      init: 0
+      lr: 0.005
+    tilt_angle_azimuth:                   # (optimizable) illumination tilt azimuth angle in radians
+      init: 0
+      lr: 0.001
+    invert_phase_contrast: false
+  apply_inverse:
+    reconstruction_algorithm: Tikhonov
+    regularization_strength: 0.01
+optimization:
+  num_iterations: 10
+  loss:
+    type: midband_power
+    midband_fractions: [0.125, 0.25]
+  log_dir: ./runs/phase_optim