Spectral Attenuation Pipeline

The core repository is a reproducible Python pipeline that turns raw depth-resolved measurement files into structured outputs for water-quality modeling. It includes data auditing, attenuation-output generation, dataset assembly, feature engineering, split generation, and derived optical summaries.

The processed datasets support two spectral configurations: a FULL range from 380 to 900 nm and a PAR range from 400 to 700 nm. Both are built from the same raw source table and exported as compact modeling datasets.

The repository also contains an application layer around the research code: a FastAPI backend and a React/Vite frontend that provide a usable interface for selected scientific analysis workflows.

Raw underwater light measurements are difficult to reuse directly for modeling because they are spread across heterogeneous field files, locations, dates, and measurement conventions.

Water-quality modeling workflows need consistent attenuation targets and aligned environmental features rather than ad hoc notebook processing.

Light-penetration analysis benefits from a reproducible way to turn field measurements into domain-specific optical metrics and downstream modeling features.

The main goal was to study how attenuation-related behavior connects to core environmental features such as chlorophyll, CDOM, and TSS within a reusable analysis workflow.

Built a public pipeline entrypoint that runs audit, attenuation-output generation, dataset, feature, split, and derived-optics stages in a fixed reproducible order.

Implemented a domain-specific processing workflow that turns raw measurement files into station-level outputs, reusable datasets, and saved visual artifacts.

Built compact FULL and PAR modeling datasets with log-transformed targets and environmental features derived from chlorophyll, CDOM, and TSS inputs.

Added reproducible split generation for both random-by-sample and holdout-by-location evaluation setups.

Added a FastAPI backend and React frontend so the scientific workflow can also be accessed through a usable web interface.

Applied domain-specific preprocessing to raw field measurements so they could be transformed into structured attenuation outputs and reusable modeling inputs.

Generated saved station-level outputs and quality-check visuals to support review of the processed optical results.

Assembled a long-form raw dataset and transformed it into compact ML tables for FULL and PAR wavelength ranges with one row per sample and one target column per wavelength.

Generated reproducible train/test exports with a fixed seed and an explicit holdout-by-location strategy to separate location generalization from random sample splits.

Derived higher-level optical and irradiance-related features for downstream analysis, interpretation, and interface-level workflows.

Reduced spectral feature correlation with PCA and fit a regression model to predict attenuation targets from chlorophyll, CDOM, and TSS inputs.

The regression model reached R² 0.86 and RMSE ≈ 0.65 (log scale) on the held-out test set — robust given the noise typical of environmental water data.

Produces a complete workflow from raw field measurements to modeling tables, split artifacts, irradiance derivatives, and reusable model exports.

A FastAPI backend and React/Vite frontend make the pipeline usable for interactive analysis — visualizing irradiance at depth and exporting results, not just running in a notebook.