repopack-output.txt

This file is a merged representation of the entire codebase, combining all repository files into a single document.
Generated by Repopack on: 2024-10-18T18:54:32.335Z

================================================================
File Summary
================================================================

Purpose:
--------
This file contains a packed representation of the entire repository's contents.
It is designed to be easily consumable by AI systems for analysis, code review,
or other automated processes.

File Format:
------------
The content is organized as follows:
1. This summary section
2. Repository information
3. Repository structure
4. Multiple file entries, each consisting of:
  a. A separator line (================)
  b. The file path (File: path/to/file)
  c. Another separator line
  d. The full contents of the file
  e. A blank line

Usage Guidelines:
-----------------
- This file should be treated as read-only. Any changes should be made to the
  original repository files, not this packed version.
- When processing this file, use the file path to distinguish
  between different files in the repository.
- Be aware that this file may contain sensitive information. Handle it with
  the same level of security as you would the original repository.

Notes:
------
- Some files may have been excluded based on .gitignore rules and Repopack's
  configuration.
- Binary files are not included in this packed representation. Please refer to
  the Repository Structure section for a complete list of file paths, including
  binary files.

Additional Info:
----------------

For more information about Repopack, visit: https://github.com/yamadashy/repopack

================================================================
Repository Structure
================================================================
feda_tools/__init__.py
feda_tools/core/analysis.py
feda_tools/core/data/__init__.py
feda_tools/core/data/array_tools.py
feda_tools/core/data/df_tools.py
feda_tools/core/data/retreival.py
feda_tools/core/data/storage.py
feda_tools/core/model.py
feda_tools/core/process.py
feda_tools/core/twodim_hist.py
feda_tools/core/utilities.py
feda_tools/gui/__main__.py
feda_tools/gui/fit23_preview.py
feda_tools/gui/process_analysis.py
feda_tools/gui/threshold_adjustment.py
feda_tools/gui/widgets.py
feda_tools/pipeline.py
LICENSE
README.md
repopack.config.json
setup.py

================================================================
Repository Files
================================================================

================
File: feda_tools/__init__.py
================
__version__ = '1.0'

================
File: feda_tools/core/analysis.py
================
from typing import Dict, List, Tuple

import seaborn as sns
import matplotlib.pyplot as plt

import numpy as np
import numpy.ma as ma
from scipy.stats import halfnorm

from . import data as dat

def running_average(data, window_size):
    window = np.ones(window_size) / window_size
    return np.convolve(data, window, mode='valid')

def flour_aniso(g_factor, intensity_para, intensity_perp, l1_japan_corr, l2_japan_corr):
    """Fluorescence Anisotropy calculation.

    See equation 7 in Kudryavtsev, V., Sikor, M., Kalinin, S., Mokranjac, D., Seidel, C.A.M. and Lamb, D.C. (2012),
    Combining MFD and PIE for Accurate Single-Pair Förster Resonance Energy Transfer Measurements. ChemPhysChem, 13: 1060-1078.
    https://doi.org/10.1002/cphc.201100822
    """
    numerator = g_factor * intensity_para - intensity_perp
    denominator = ((1 - 3 * l2_japan_corr) * g_factor * intensity_para +
                   (2 - 3 * l1_japan_corr) * intensity_perp)
    return numerator / denominator

def interphoton_arrival_times(all_macro_times, all_micro_times, macro_res, micro_res):
    # - Each detected photon has a time of detection encoded by the macro time + the micro time. **all_macro_times** and **all_micro_times** are arrays whose index is represents the detected photons in order of detection, while the value represents the associated macro or micro time for each photon.
    # - **macro_res** and **micro_res** represent the resolution of the macro and micro times in seconds.
    # - The **macro time** indicates the time in units of **macro_res** that the excitation laser was last fired directly before this photon was detected.
    # - The **micro time** indicates the amount of time in units of **micro_res** that has elapsed since the excitation laser was last fired at which the photon was detected, i.e. it's the amount of time elapsed from the macro time at which the photon was detected.
    # - The interphoton arrival time is calculated by iterating through **all_macro_times** and **all_micro_times** and calculating the time elapsed between each photon detection event.
    
    arr_size = len(all_macro_times) - 1
    photon_time_intervals = np.zeros(arr_size, dtype=np.float64)
    for i in range(arr_size):
        photon_1 = (all_macro_times[i]*macro_res) + (all_micro_times[i]*micro_res)
        photon_2 = (all_macro_times[i+1]*macro_res) + (all_micro_times[i+1]*micro_res)
        photon_time_intervals[i] = (photon_2 - photon_1)*1000  # Convert to ms
    photon_ids = np.arange(1, arr_size + 1)
    return photon_time_intervals, photon_ids

def estimate_background_noise(logrunavg, bins_y):
    counts_logrunavg, bins_logrunavg, _ = plt.hist(logrunavg, bins=bins_y, alpha=0.6, color='r')
    plt.close()  # Close the plot to prevent it from displaying during function call
    index_max = np.argmax(counts_logrunavg)
    noise_mean = bins_logrunavg[index_max]*0.95
    filtered_logrunavg = ma.masked_less(logrunavg, noise_mean).compressed()
    mu, std = halfnorm.fit(filtered_logrunavg)
    return mu, std, noise_mean, filtered_logrunavg, bins_logrunavg

================
File: feda_tools/core/data/__init__.py
================
from .retreival import *
from .storage import *
from .df_tools import *
from .array_tools import *

================
File: feda_tools/core/data/array_tools.py
================
import numpy.ma as ma

### define function for extracting the unmasked segments from the thresholded data.
def extract_greater(data, threshold_value):
    filtered_values = ma.masked_greater(data, threshold_value)
    burst_index = extract_unmasked_indices(filtered_values)
    return burst_index, filtered_values

def extract_unmasked_indices(masked_array):
    unmasked_indices_lists = []
    current_indices = []

    # iterate through masked array and collect unmasked index segments
    for i, value in enumerate(masked_array):
        if ma.is_masked(value):
            if current_indices:
                unmasked_indices_lists.append(current_indices)
                current_indices = []
        else:
            current_indices.append(i)

    # handle the last segment
    if current_indices:
        unmasked_indices_lists.append(current_indices)

    return unmasked_indices_lists

================
File: feda_tools/core/data/df_tools.py
================
import pandas as pd

def filter_burstids(df, bid_path):
    
    # get bid_df to use as a filter on the passed df.
    bid_df = get_bid_df(bid_path)
    
    # Filter df using bid_df.
    result_df = df.merge(bid_df, on=['First Photon', 'Last Photon', 'First File'], how='inner')
    
    return result_df

# read all of the burst id files in the selected directory and create a dataframe storing 
# their First Photon, Last Photon, and First File values row-wise.

def get_bid_df(bid_path):

    # create dfs list to append each bst file dataframe when reading the directory
    dfs = []
    
    # iterate through the bst files in the specified directory
    for file in sorted((bid_path).glob('*.%s' % 'bst')):
        
        # read each file and create a dataframe
        file_df = pd.read_csv(file, sep ='\t', header=None)
        
        # rename the unnammed columns
        file_df.rename(columns={0: "First Photon", 1: "Last Photon"}, inplace=True)
        
        # get the filename so we can assoc a file to the First File column
        filename = os.path.basename(file).split('.')[0]
        
        # process the filename so that it matches the format in the bur file
        filename = filename.replace("_0", "")
        filename = filename + ".ptu"

        # assign this files First and Last Photon data to the associated file.
        file_df['First File'] = filename
        
        # append the burst file dataframe to a list, will concatenate them all after loop.
        dfs.append(file_df)
        
    # concatenate all the dfs into one 
    flphotons = pd.concat(dfs, axis = 0, ignore_index = True)

    return flphotons

================
File: feda_tools/core/data/retreival.py
================
import os
import tttrlib
import fnmatch

def get_ptu_files(directory):
    ptu_files = []
    for file in os.listdir(directory):
        if fnmatch.fnmatch(file, '*.ptu'):
            ptu_files.append(file)
    return ptu_files

def load_ptu_files(file_ptu, file_irf, file_bkg):
    data_ptu = tttrlib.TTTR(file_ptu, 'PTU')
    data_irf = tttrlib.TTTR(file_irf, 'PTU')
    data_bkg = tttrlib.TTTR(file_bkg, 'PTU')
    return data_ptu, data_irf, data_bkg

================
File: feda_tools/core/data/storage.py
================
import os

def save_results(output_directory, file_path, bi4_bur_df, bg4_df):
    bur_filename = os.path.splitext(os.path.basename(str(file_path)))[0]
    bur_filepath = os.path.join(output_directory, bur_filename) + ".bur"
    bi4_bur_df.to_csv(bur_filepath, sep='\t', index=False, float_format='%.6f')
    bg4_filepath = os.path.join(output_directory, bur_filename) + ".bg4"
    bg4_df.to_csv(bg4_filepath, sep='\t', index=False, float_format='%.6f')

================
File: feda_tools/core/model.py
================
import tttrlib
import numpy as np

def setup_fit23(num_bins, macro_res, counts_irf_nb, g_factor, l1_japan_corr, l2_japan_corr):
    dt = 25000/num_bins/1000
    period = 1/(macro_res*np.power(10, 6))
    fit23 = tttrlib.Fit23(
        dt=dt,
        irf=counts_irf_nb,
        background=np.ones_like(counts_irf_nb)*0.002,
        period=period,
        g_factor=g_factor,
        l1=l1_japan_corr,
        l2=l2_japan_corr,
        convolution_stop=10,
        p2s_twoIstar_flag=True
    )
    return fit23

================
File: feda_tools/core/process.py
================
import numpy as np
import pandas as pd
from tqdm.auto import tqdm

from feda_tools.core import analysis as an

# This function realistically should be broken up, its a spaghetti code mess that I'll or someone else will
# have to come back to and fix - Alex K 10-18-2024
def process_single_burst(
    burst, all_macro_times, all_micro_times, routing_channels, macro_res, micro_res,
    min_photon_count, bg4_micro_time_min, bg4_micro_time_max, g_factor, l1_japan_corr,
    l2_japan_corr, bg4_bkg_para, bg4_bkg_perp, fit23, initial_fit_params
):
    if len(burst) <= min_photon_count:
        return None, None

    first_photon = burst[0]
    last_photon = burst[-1]
    lp_time = all_macro_times[last_photon]*macro_res + all_micro_times[last_photon]*micro_res
    fp_time = all_macro_times[first_photon]*macro_res + all_micro_times[first_photon]*micro_res
    lp_time_ms = lp_time*1000
    fp_time_ms = fp_time*1000
    duration = lp_time_ms - fp_time_ms
    macro_times = all_macro_times[burst]*macro_res*1000
    mean_macro_time = np.mean(macro_times)
    num_photons = len(burst)
    count_rate = num_photons / duration if duration != 0 else np.nan
    list_of_indexes = burst
    mask_channel_0 = routing_channels[list_of_indexes] == 0
    mask_channel_2 = routing_channels[list_of_indexes] == 2
    indexes_channel_0 = np.array(list_of_indexes)[mask_channel_0]
    indexes_channel_2 = np.array(list_of_indexes)[mask_channel_2]

    if len(indexes_channel_0) > 0 and len(indexes_channel_2) > 0:
        first_green_photon = min(np.min(indexes_channel_0), np.min(indexes_channel_2))
        last_green_photon = max(np.max(indexes_channel_0), np.max(indexes_channel_2))
    elif len(indexes_channel_0) >= 2:
        first_green_photon = np.min(indexes_channel_0)
        last_green_photon = np.max(indexes_channel_0)
    elif len(indexes_channel_2) >= 2:
        first_green_photon = np.min(indexes_channel_2)
        last_green_photon = np.max(indexes_channel_2)
    else:
        first_green_photon = None
        last_green_photon = None

    if first_green_photon is not None and last_green_photon is not None:
        lgp_time = all_macro_times[last_green_photon]*macro_res + all_micro_times[last_green_photon]*micro_res
        fgp_time = all_macro_times[first_green_photon]*macro_res + all_micro_times[first_green_photon]*micro_res
        lgp_time_ms = lgp_time*1000
        fgp_time_ms = fgp_time*1000
        duration_green = lgp_time_ms - fgp_time_ms
    else:
        duration_green = np.nan

    macro_times_ch0 = all_macro_times[indexes_channel_0]*macro_res*1000
    macro_times_ch2 = all_macro_times[indexes_channel_2]*macro_res*1000
    combined_macro_times = np.concatenate([macro_times_ch0, macro_times_ch2], axis=0)
    mean_macro_time_green = np.mean(combined_macro_times) if len(combined_macro_times) > 0 else np.nan
    num_photons_gr = len(indexes_channel_0) + len(indexes_channel_2)
    count_rate_gr = num_photons_gr / duration_green if duration_green != 0 else np.nan

    bur_new_row = {
        'First Photon': [first_photon],
        'Last Photon': [last_photon],
        'Duration (ms)': [duration],
        'Mean Macro Time (ms)': [mean_macro_time],
        'Number of Photons': [num_photons],
        'Count Rate (kHz)': [count_rate],
        'Duration (green) (ms)': [duration_green],
        'Mean Macro Time (green) (ms)': [mean_macro_time_green],
        'Number of Photons (green)': [num_photons_gr],
        'Green Count Rate (kHz)': [count_rate_gr]
    }
    bi4_bur_df = pd.DataFrame.from_dict(bur_new_row)

    bg4_channel_2_photons = [
        index for index in burst if routing_channels[index] == 2 and
        bg4_micro_time_min < all_micro_times[index] < bg4_micro_time_max
    ]
    bg4_channel_2_count = len(bg4_channel_2_photons)

    bg4_channel_0_photons = [
        index for index in burst if routing_channels[index] == 0 and
        bg4_micro_time_min < all_micro_times[index] < bg4_micro_time_max
    ]
    bg4_channel_0_count = len(bg4_channel_0_photons)

    bg4_total_count = bg4_channel_2_count + bg4_channel_0_count
    bg4_rexp = an.flour_aniso(
        g_factor, bg4_channel_2_count, bg4_channel_0_count,
        l1_japan_corr, l2_japan_corr
    )
    bg4_rscat = an.flour_aniso(
        g_factor,
        bg4_channel_2_count - bg4_bkg_para,
        bg4_channel_0_count - bg4_bkg_perp,
        l1_japan_corr,
        l2_japan_corr
    )
    counts = np.array([bg4_channel_0_count, bg4_channel_2_count])

    # Use initial_fit_params as starting values
    x0 = initial_fit_params
    fixed = np.array([0, 0, 1, 0])

    try:
        r2 = fit23(data=counts, initial_values=x0, fixed=fixed, include_model=True)
        fit_tau, fit_gamma, fit_r0, fit_rho = r2['x'][:4]
        fit_softbifl = r2['x'][4] if len(r2['x']) > 4 else np.nan
        fit_2istar = r2['x'][5] if len(r2['x']) > 5 else np.nan
        fit_rs_scatter = r2['x'][6] if len(r2['x']) > 6 else np.nan
        fit_rs_exp = r2['x'][7] if len(r2['x']) > 7 else np.nan
    except Exception as e:
        print(f"Fit23 error for burst {first_photon}-{last_photon}: {e}")
        return None, None

    bg4_new_row = {
        'Ng-p-all': [bg4_channel_2_count],
        'Ng-s-all': [bg4_channel_0_count],
        'Number of Photons (fit window) (green)': [bg4_total_count],
        'r Scatter (green)': [bg4_rscat],
        'r Experimental (green)': [bg4_rexp],
        'Fit tau': [fit_tau],
        'Fit gamma': [fit_gamma],
        'Fit r0': [fit_r0],
        'Fit rho': [fit_rho],
        'Fit softbifl': [fit_softbifl],
        'Fit 2I*': [fit_2istar],
        'Fit rs_scatter': [fit_rs_scatter],
        'Fit rs_exp': [fit_rs_exp]
    }
    bg4_df = pd.DataFrame.from_dict(bg4_new_row)

    return bi4_bur_df, bg4_df

================
File: feda_tools/core/twodim_hist.py
================
"""
Author - Frank Duffy

"""

import argparse
import os
import yaml
import pandas as pd
from matplotlib import colors
from matplotlib.ticker import PercentFormatter
import matplotlib.pyplot as plt
import numpy as np

calc_list = [
    "Mean Macro Time (sec)",
    "Sg/Sr (prompt)",
    "S(prompt)/S(total)"
]

def get_plot_dict(yaml_file):
    """
    gets the plots specified by the user in the provided yaml file.
    """

    # Conversts yaml doc to python object
    plot_dict = yaml.safe_load(yaml_file)
    return plot_dict

def arg_check(arg):
    """
    checks if the arguments provided by the user correspond to paths that exist
    """
    path = str(arg)

    if os.path.exists(path):
        return path
    else:
        raise argparse.ArgumentTypeError(path + ' could not be found. ' + 
                                         'Check for typos or for errors in your relative path string')

def parse_args(args):
    """
    parse the arguments provided by the user and return them to the main program
    """
    parser = argparse.ArgumentParser()
    parser.add_argument('data_folder', type=arg_check)
    parser.add_argument('plot_file', type=arg_check)
    parsed_args = parser.parse_args(args)
    print((parsed_args.data_folder, parsed_args.plot_file))

    return parsed_args.data_folder, parsed_args.plot_file

def get_data(data_folder):
    print("Getting data in " + data_folder)
    # print(os.getcwd())
    df_list = []
    for file in os.listdir(data_folder):
        df_list.append(pd.read_csv(data_folder + "\\" + file, sep = '\t'))
    df = pd.concat(df_list)
    # print(df)
    return df

def get_calc(label, data_folder):
    """
    perform the requisite calculation on the data frame and return it
    """
    
    if label == "Mean Macro Time (sec)":
        
        # data_folder = data_folder 
        df = get_data(data_folder + "\\bi4_bur" )
        df["Mean Macro Time (ms)"] = df["Mean Macro Time (ms)"].div(1000)
        df = df.rename(columns={"Mean Macro Time (ms)": "Mean Macro Time (sec)"})
        return df
    
    elif label == "Sg/Sr (prompt)":
        df = get_data(data_folder + "\\bi4_bur")
        df[label] = df["Green Count Rate (KHz)"].div(df["S prompt red (kHz) | 0-200"])
        # df[label] = np.log(df[label])
        return df

def clean_data(df):
    if "Number of Photons" in df.columns:
        df = df.loc[df["Number of Photons"] > 0]
    elif "Number of Photons (fit window) (green)" in df.columns:
        df = df.loc[df["Number of Photons (fit window) (green)"] > 0]
    elif "Number of Photons (fit window) (red)" in df.columns:
        df = df.loc[df["Number of Photons (fit window) (red)"] > 0]
    elif "Number of Photons (fit window) (yellow)" in df.columns:
        df = df.loc[df["Number of Photons (fit window) (yellow)"] > 0]
    
    if "Unnamed: 14" in df.columns:
        df.drop(labels="Unnamed: 14", axis = 1, inplace=True)
        # df = df[df["Tau (yellow)"].between(1,6)]
        # df = df[df["r Scatter (yellow)"].between(-0.2,1.5)]
    
    # if "TGX_TRR" in df.columns:
    #     df = df[df["TGX_TRR"].between(-5,5)]

    print(df)
    df.replace([np.inf, -np.inf], np.nan, inplace =True)
    print(df)
    df.dropna(inplace = True)
    print(df)

    return df

def make_plot(x, y, xlabel, ylabel, xrange, yrange, bins):
    # x is an array-like 
    # y is an array-like
    # xlabel is a string
    # ylabel is a string
    # xrange is either the string "auto" or a dict {"min": <your float/int here>, "max": <your float/int here>}
    # yrange is either the string "auto" or a dict {"min": <your float/int here>, "max": <your float/int here>}
    # bins is a dict {"x": <your int here>, "y": <your int here>}

    n_binsx = bins["x"]
    n_binsy = bins["y"] 
    c_map = 'gist_ncar_r'

    # Create a Figure, which doesn't have to be square.
    fig = plt.figure(layout='constrained')
    
    # Create the main axes, leaving 25% of the figure space at the top and on the
    # right to position marginals.
    ax = fig.add_gridspec(top=0.75, right=0.75).subplots()
    
    # The main axes' aspect can be fixed.
    ax.set(aspect="auto")
    ax_histx = ax.inset_axes([0, 1.05, 1.0, 0.25], sharex=ax)
    ax_histy = ax.inset_axes([1.05, 0, 0.25, 1], sharey=ax)

    # Draw the scatter plot and marginals.
    # no labels
    ax_histx.tick_params(axis="x", labelbottom=False)
    ax_histy.tick_params(axis="y", labelleft=False)

    if xrange != "auto":
        n_binsx = np.linspace(xrange["min"], xrange["max"], num=n_binsx)

    if yrange != "auto":
        n_binsy = np.linspace(yrange["min"], yrange["max"], num=n_binsy)
 
    if ylabel == "Sg/Sr (prompt)":
        n_binsy = np.geomspace(np.min(y), np.max(y), num=n_binsy)
        plt.yscale("log")
 
    # the 2d hist plot:
    h = ax.hist2d(x, y, bins = [n_binsx, n_binsy], cmap = c_map)
    hist_values_2d = h[0]
    mappable = h[3]
    fig.colorbar(mappable, ax=ax, location='left')


    # now determine nice limits by hand:
    binwidth = 0.25
    xymax = max(np.max(np.abs(x)), np.max(np.abs(y)))
    lim = (int(xymax/binwidth) + 1) * binwidth

    bins = np.arange(-lim, lim + binwidth, binwidth)
    ax_histx.hist(x, bins=n_binsx)
    ax_histy.hist(y, bins=n_binsy, orientation='horizontal')

    ax.set_xlabel(xlabel, fontsize = 20)
    ax.set_ylabel(ylabel, fontsize = 20)

    return fig, ax, hist_values_2d, ax_histx, ax_histy

def make_2dhist(args=None):
    
    data_folder, plot_file = parse_args(args)
    
    with open(plot_file) as yaml_file:
        plot_dict = get_plot_dict(yaml_file)
    
    for plot in plot_dict:
        
        xlabel = plot_dict[plot]['xlabel']
        ylabel = plot_dict[plot]['ylabel']
        xrange = plot_dict[plot]['xrange']
        
        print("Plotting (" + xlabel + ", " + ylabel + ")")

        # check if coordinate is a calculation
        if xlabel in calc_list:
            print("Calculating " + xlabel)
            x_df = get_calc(xlabel, data_folder)
        else:
            xfolder = plot_dict[plot]['xfolder']
            print("Getting " + xlabel + " from " + xfolder)
            xdata_folder = data_folder + "\\" + xfolder
            x_df = get_data(xdata_folder)

        if ylabel in calc_list:
            print("Calculating " + ylabel)
            y_df = get_calc(ylabel, data_folder)
        else:
            yfolder = plot_dict[plot]['yfolder']
            print("Getting " + ylabel + " from " + yfolder)
            ydata_folder = data_folder + "\\" + yfolder
            y_df = get_data(ydata_folder)

        # clean the data i.e. remove photon counts == 0, ignore NaN and inf, etc.
        # x_df = clean_data(x_df)
        # y_df = clean_data(y_df)

        print(x_df[xlabel])
        print(y_df[ylabel])

        print(x_df)

        # dataset = pd.concat([x_df, y_df[ylabel]], axis = 1)
        if np.array_equal(x_df, y_df):
            # same dataset, just take x_df
            print("same data set")
            dataset = x_df
        elif ylabel in x_df.columns:
            # conflict, assume we'd rather have the ylabel column in y_df
            print(ylabel + " in x_df")
            x_df.drop(ylabel, axis = 1, inplace=True)
            dataset = x_df.join(y_df[ylabel])
        else:
            # no conflicts, just join
            print("***No conflicts***")
            dataset = x_df.join(y_df[ylabel])
        
        print(dataset)

        dataset = clean_data(dataset)
        print(dataset)

        make_plot(dataset[xlabel].to_numpy(), dataset[ylabel].to_numpy(), xlabel, xrange, ylabel)
    
    plt.show()

================
File: feda_tools/core/utilities.py
================
from typing import Dict, List, Tuple

import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pathlib
import os
import tqdm
import tttrlib
import fnmatch

def update_tttr_dict(
    df: pd.DataFrame,
    data_path: pathlib.Path,
    tttrs: Dict[str, tttrlib.TTTR] = dict(),
    file_type: str = "PTU"
):
    for ff, fl in zip(df['First File'], df['Last File']):
        try:
            tttr = tttrs[ff]
        except KeyError:
            fn = str(data_path / ff)
            tttr = tttrlib.TTTR(fn, file_type)
            tttrs[ff] = tttr    
    return tttrs

def read_analysis(
    paris_path : pathlib.Path,
    paths: List[str] = ['bg4', 'bi4_bur'], #
    file_endings: List[str] = ['bg4', 'bur'],  # 
    file_type: str = "PTU"
) -> (pd.DataFrame, Dict[str, tttrlib.TTTR]):
    
    info_path = paris_path / 'Info'
    data_path = paris_path.parent

    dfs = list()
    for path, ending in zip(paths, file_endings):
        frames = list()
        for fn in sorted((paris_path / path).glob('*.%s' % ending)):
            df = pd.read_csv(fn, sep='\t')
            df.drop(df.columns[df.columns.str.contains('unnamed',case = False)],axis = 1, inplace = True)
            frames.append(df)
        path_df = pd.concat(frames)
        dfs.append(path_df)
    df = pd.concat(dfs, axis=1)
    
#     df = df.dropna()
    
    # Loop through each column and attempt to convert to numeric
    for column in df.columns:
        try:
            df[column] = pd.to_numeric(df[column])
        except ValueError:
            # Handle exceptions, e.g., if the column contains non-numeric values
            print(f"Could not convert {column} to numeric")
    tttrs = dict()
    update_tttr_dict(df, data_path, tttrs, file_type)
    return df, tttrs


def extract_unmasked_indices(masked_array):
    unmasked_indices_lists = []
    current_indices = []
    for i, value in enumerate(masked_array):
        if np.ma.is_masked(value):
            if current_indices:
                unmasked_indices_lists.append(current_indices)
                current_indices = []
        else:
            current_indices.append(i)
    if current_indices:
        unmasked_indices_lists.append(current_indices)
    return unmasked_indices_lists

================
File: feda_tools/gui/__main__.py
================
import sys
from PyQt6 import QtWidgets
from feda_tools.gui.threshold_adjustment import ThresholdAdjustmentWindow

class MainApplication(QtWidgets.QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle('FEDA Tools GUI')
        self.init_ui()

    def init_ui(self):
        # Set ThresholdAdjustmentWindow as the central widget
        self.threshold_window = ThresholdAdjustmentWindow()
        self.setCentralWidget(self.threshold_window)
        self.show()

def main():
    app = QtWidgets.QApplication(sys.argv)
    main_window = MainApplication()
    sys.exit(app.exec())

if __name__ == '__main__':
    main()

================
File: feda_tools/gui/fit23_preview.py
================
from PyQt6 import QtWidgets, QtCore
from .widgets import MatplotlibCanvas
from feda_tools.core import model
import numpy as np

class Fit23PreviewWindow(QtWidgets.QWidget):
    analysis_started = QtCore.pyqtSignal()

    def __init__(self, data_ptu, data_irf, burst_index, chunk_size, fit_params, output_directory):
        super().__init__()
        self.data_ptu = data_ptu
        self.data_irf = data_irf
        self.burst_index = burst_index
        self.chunk_size = chunk_size
        self.fit_params = fit_params
        self.output_directory = output_directory  # Save the output directory

        self.init_ui()
        self.thread = None
        self.worker = None

    def init_ui(self):
        self.setWindowTitle('Fit23 Preview')
        layout = QtWidgets.QVBoxLayout()

        # Matplotlib canvas
        self.canvas = MatplotlibCanvas(self, width=5, height=4, dpi=100)
        layout.addWidget(self.canvas)

        # Fit23 Parameters Input
        params_layout = QtWidgets.QFormLayout()
        
        self.tau_input = QtWidgets.QDoubleSpinBox()
        self.tau_input.setRange(0.0, 100.0)
        self.tau_input.setDecimals(4)
        self.tau_input.setValue(3.03)  # Default value
        self.tau_input.valueChanged.connect(self.update_preview)
        
        self.gamma_input = QtWidgets.QDoubleSpinBox()
        self.gamma_input.setRange(0.0, 100.0)
        self.gamma_input.setDecimals(4)
        self.gamma_input.setValue(0.02)  # Default value
        self.gamma_input.valueChanged.connect(self.update_preview)
        
        self.r0_input = QtWidgets.QDoubleSpinBox()
        self.r0_input.setRange(0.0, 100.0)
        self.r0_input.setDecimals(4)
        self.r0_input.setValue(0.38)  # Default value
        self.r0_input.valueChanged.connect(self.update_preview)
        
        self.rho_input = QtWidgets.QDoubleSpinBox()
        self.rho_input.setRange(0.0, 100.0)
        self.rho_input.setDecimals(4)
        self.rho_input.setValue(1.64)  # Default value
        self.rho_input.valueChanged.connect(self.update_preview)

        params_layout.addRow('Initial Tau:', self.tau_input)
        params_layout.addRow('Initial Gamma:', self.gamma_input)
        params_layout.addRow('Initial r0:', self.r0_input)
        params_layout.addRow('Initial rho:', self.rho_input)

        layout.addLayout(params_layout)

        # Run Fit23 Button
        run_fit_button = QtWidgets.QPushButton('Run Fit23 with Current Parameters')
        run_fit_button.clicked.connect(self.run_fit23)
        layout.addWidget(run_fit_button)

        # Run Analysis button
        self.run_button = QtWidgets.QPushButton('Run Full Analysis')
        layout.addWidget(self.run_button)
        self.run_button.clicked.connect(self.start_analysis)

        self.setLayout(layout)
        self.plot_fit23(initial_plot=True)

    def plot_fit23(self, initial_plot=False):
        # Prepare data for the entire chunk
        num_bins = 128

        # Get micro_times from data_ptu
        all_micro_times = self.data_ptu.micro_times

        # Create histogram counts for the chunk
        counts, _ = np.histogram(all_micro_times, bins=num_bins)

        # Prepare counts_irf from data_irf
        counts_irf, _ = np.histogram(self.data_irf.micro_times, bins=num_bins)
        counts_irf_nb = counts_irf.copy()
        counts_irf_nb[0:3] = 0
        counts_irf_nb[10:66] = 0
        counts_irf_nb[74:128] = 0

        # Setup fit23 with default or provided parameters
        macro_res = self.data_ptu.get_header().macro_time_resolution
        g_factor = 1.04
        l1_japan_corr = 0.0308
        l2_japan_corr = 0.0368

        if initial_plot:
            initial_fit_params = np.array([
                self.tau_input.value(),
                self.gamma_input.value(),
                self.r0_input.value(),
                self.rho_input.value()
            ])
            self.fit_params = initial_fit_params.copy()
        else:
            self.fit_params = np.array([
                self.tau_input.value(),
                self.gamma_input.value(),
                self.r0_input.value(),
                self.rho_input.value()
            ])


        self.fit23_model = model.setup_fit23(
            num_bins, macro_res, counts_irf_nb, g_factor, l1_japan_corr, l2_japan_corr
        )

        # Perform fitting
        x0 = self.fit_params  # Use current fit_params
        fixed = np.array([0, 0, 1, 0])
        try:
            r2 = self.fit23_model(data=counts, initial_values=x0, fixed=fixed, include_model=True)
            self.fit_params = r2['x'][:4]
        except Exception as e:
            QtWidgets.QMessageBox.warning(self, 'Fit23 Error', f'An error occurred during Fit23:\n{e}')
            return

        # Plot data and model
        self.canvas.axes.clear()
        self.canvas.axes.semilogy(counts / np.max(counts), label='Data')
        self.canvas.axes.semilogy(counts_irf / np.max(counts_irf), label='IRF')
        self.canvas.axes.semilogy(self.fit23_model.model / np.max(self.fit23_model.model), label='Model')

        # Set y-axis limits
        self.canvas.axes.set_ylim(0.001, 1)

        # Set labels and legend
        self.canvas.axes.set_ylabel('log(Counts)')
        self.canvas.axes.set_xlabel('Channel Number')
        self.canvas.axes.legend()

        # Draw the canvas
        self.canvas.draw()
    
    def update_preview(self):
        self.plot_fit23()

    def run_fit23(self):
        # Re-plot Fit23 with user-defined initial parameters
        self.plot_fit23()

    def start_analysis(self):
        # Proceed to full analysis using the latest fit_params
        from .process_analysis import ProcessAnalysisWindow

        self.process_window = ProcessAnalysisWindow(
            self.data_ptu, self.data_irf, self.burst_index, self.chunk_size, self.fit_params, self.output_directory
        )
        self.process_window.show()
        self.analysis_started.emit()
        self.close()

================
File: feda_tools/gui/process_analysis.py
================
from PyQt6 import QtWidgets, QtCore
from PyQt6.QtCore import Qt, QThread, pyqtSignal
from .widgets import MatplotlibCanvas
from feda_tools.core import process as proc
from feda_tools.core import data as dat
from feda_tools.core import model
from feda_tools.data import save_results
import numpy as np
import pandas as pd

class Worker(QtCore.QObject):
    progress_update = pyqtSignal(int)
    plot_update = pyqtSignal()
    finished = pyqtSignal()
    error_occurred = pyqtSignal(str)

    def __init__(self, data_ptu, data_irf, burst_index, chunk_size, fit_params, update_interval):
        super().__init__()
        self.data_ptu = data_ptu
        self.data_irf = data_irf
        self.burst_index = burst_index
        self.chunk_size = chunk_size
        self.fit_params = fit_params
        self.update_interval = update_interval
        self.sg_sr = []
        self.mean_macro_time = []
        self.tau_values = []
        self.r_s_values = []
        self.is_running = True

    @QtCore.pyqtSlot()
    def run(self):
        try:
            # Example parameters, adjust as needed
            min_photon_count = 60
            bg4_micro_time_min = 0
            bg4_micro_time_max = 12499
            g_factor = 1.04
            l1_japan_corr = 0.0308
            l2_japan_corr = 0.0368
            bg4_bkg_para = 0
            bg4_bkg_perp = 0
            num_bins = 128

            # Prepare data
            total_bursts = len(self.burst_index)
            self.progress_update.emit(0)

            # Initialize fit23 model
            counts_irf, _ = np.histogram(self.data_irf.micro_times, bins=num_bins)
            counts_irf_nb = counts_irf.copy()
            counts_irf_nb[0:3] = 0
            counts_irf_nb[10:66] = 0
            counts_irf_nb[74:128] = 0
            macro_res = self.data_ptu.get_header().macro_time_resolution
            fit23_model = model.setup_fit23(
                num_bins, macro_res, counts_irf_nb, g_factor, l1_japan_corr, l2_japan_corr
            )

            # Processing bursts
            for i, burst in enumerate(self.burst_index, 1):
                if not self.is_running:
                    break  # Allow for future extension to stop processing

                # Process each burst individually
                bi4_bur_df, bg4_df = proc.process_single_burst(
                    burst,
                    self.data_ptu.macro_times,
                    self.data_ptu.micro_times,
                    self.data_ptu.routing_channels,
                    macro_res,
                    self.data_ptu.get_header().micro_time_resolution,
                    min_photon_count,
                    bg4_micro_time_min,
                    bg4_micro_time_max,
                    g_factor,
                    l1_japan_corr,
                    l2_japan_corr,
                    bg4_bkg_para,
                    bg4_bkg_perp,
                    fit23_model,
                    self.fit_params
                )

                if bi4_bur_df is not None and bg4_df is not None:
                    try:
                        sg_sr_value = bg4_df['Ng-p-all'].values[0] / bg4_df['Ng-s-all'].values[0]
                        tau_value = bg4_df['Fit tau'].values[0]
                        r_s_value = bg4_df['Fit rs_scatter'].values[0]
                        mean_macro = bi4_bur_df['Mean Macro Time (ms)'].values[0]

                        self.sg_sr.append(sg_sr_value)
                        self.mean_macro_time.append(mean_macro)
                        self.tau_values.append(tau_value)
                        self.r_s_values.append(r_s_value)
                    except Exception as e:
                        self.error_occurred.emit(f"Error extracting plot data for burst {i}: {e}")

                # Update progress bar
                self.progress_update.emit(i)

                # Update plots every n bursts
                if i % self.update_interval == 0:
                    self.plot_update.emit()

            # Final plot update
            self.plot_update.emit()
            self.finished.emit()

        except Exception as e:
            self.error_occurred.emit(str(e))
            self.finished.emit()

    def stop(self):
        self.is_running = False

class ProcessAnalysisWindow(QtWidgets.QWidget):
    def __init__(self, data_ptu, data_irf, burst_index, chunk_size, fit_params, output_directory):
        super().__init__()
        self.data_ptu = data_ptu
        self.data_irf = data_irf
        self.burst_index = burst_index
        self.chunk_size = chunk_size
        self.fit_params = fit_params
        self.output_directory = output_directory

        self.init_ui()
        self.thread = None
        self.worker = None

        # Data containers for plots
        self.sg_sr = []
        self.mean_macro_time = []
        self.tau_values = []
        self.r_s_values = []

    def init_ui(self):
        self.setWindowTitle('Full Analysis')
        layout = QtWidgets.QVBoxLayout()

        # Progress bar
        self.progress_bar = QtWidgets.QProgressBar()
        layout.addWidget(self.progress_bar)

        # Update interval input
        interval_layout = QtWidgets.QHBoxLayout()
        interval_label = QtWidgets.QLabel('Update Interval (bursts):')
        self.interval_input = QtWidgets.QSpinBox()
        self.interval_input.setRange(1, 1000)
        self.interval_input.setValue(10)  # Default update interval
        interval_layout.addWidget(interval_label)
        interval_layout.addWidget(self.interval_input)
        layout.addLayout(interval_layout)

        # Start Analysis Button
        self.start_button = QtWidgets.QPushButton('Start Full Analysis')
        self.start_button.clicked.connect(self.run_full_analysis)
        layout.addWidget(self.start_button)

        # Plot area
        plot_layout = QtWidgets.QHBoxLayout()
        self.canvas1 = MatplotlibCanvas(self, width=5, height=4, dpi=100)
        self.canvas2 = MatplotlibCanvas(self, width=5, height=4, dpi=100)
        self.canvas3 = MatplotlibCanvas(self, width=5, height=4, dpi=100)

        plot_layout.addWidget(self.canvas1)
        plot_layout.addWidget(self.canvas2)
        plot_layout.addWidget(self.canvas3)

        layout.addLayout(plot_layout)

        self.setLayout(layout)

    def run_full_analysis(self):
        if self.worker and self.worker.isRunning():
            QtWidgets.QMessageBox.warning(self, 'Processing', 'Analysis is already running.')
            return

        self.start_button.setEnabled(False)
        self.update_interval = self.interval_input.value()

        # Initialize Worker and Thread
        self.thread = QThread()
        self.worker = Worker(
            self.data_ptu,
            self.data_irf,
            self.burst_index,
            self.chunk_size,
            self.fit_params,
            self.update_interval
        )
        self.worker.moveToThread(self.thread)

        # Connect signals and slots
        self.thread.started.connect(self.worker.run)
        self.worker.progress_update.connect(self.update_progress)
        self.worker.plot_update.connect(self.update_plots)
        self.worker.finished.connect(self.analysis_finished)
        self.worker.error_occurred.connect(self.handle_error)
        self.worker.finished.connect(self.thread.quit)
        self.worker.finished.connect(self.worker.deleteLater)
        self.thread.finished.connect(self.thread.deleteLater)

        # Start the thread
        self.thread.start()

    @QtCore.pyqtSlot(int)
    def update_progress(self, value):
        self.progress_bar.setValue(value)

    @QtCore.pyqtSlot()
    def update_plots(self):
        # sg/sr vs Mean Macro Time
        self.canvas1.axes.clear()
        self.canvas1.axes.scatter(self.worker.mean_macro_time, self.worker.sg_sr, s=10, alpha=0.7)
        self.canvas1.axes.set_xlabel('Mean Macro Time (ms)')
        self.canvas1.axes.set_ylabel('sg/sr')
        self.canvas1.axes.set_title('sg/sr vs Mean Macro Time')
        self.canvas1.axes.grid(True)
        self.canvas1.draw()

        # sg/sr vs Tau
        self.canvas2.axes.clear()
        self.canvas2.axes.scatter(self.worker.tau_values, self.worker.sg_sr, s=10, alpha=0.7, color='green')
        self.canvas2.axes.set_xlabel('Tau')
        self.canvas2.axes.set_ylabel('sg/sr')
        self.canvas2.axes.set_title('sg/sr vs Tau')
        self.canvas2.axes.grid(True)
        self.canvas2.draw()

        # r_s vs Tau
        self.canvas3.axes.clear()
        self.canvas3.axes.scatter(self.worker.tau_values, self.worker.r_s_values, s=10, alpha=0.7, color='red')
        self.canvas3.axes.set_xlabel('Tau')
        self.canvas3.axes.set_ylabel('r_s')
        self.canvas3.axes.set_title('r_s vs Tau')
        self.canvas3.axes.grid(True)
        self.canvas3.draw()

    @QtCore.pyqtSlot()
    def analysis_finished(self):
        self.start_button.setEnabled(True)
        QtWidgets.QMessageBox.information(self, 'Success', 'Full analysis completed successfully.')
        save_results()
        

    @QtCore.pyqtSlot(str)
    def handle_error(self, message):
        QtWidgets.QMessageBox.warning(self, 'Error', f'An error occurred during analysis:\n{message}')
        self.start_button.setEnabled(True)

================
File: feda_tools/gui/threshold_adjustment.py
================
from PyQt6 import QtWidgets, QtCore
from .widgets import MatplotlibCanvas
from feda_tools.core import analysis as an
from feda_tools.core import data as dat
import numpy as np

class ThresholdAdjustmentWindow(QtWidgets.QWidget):
    threshold_changed = QtCore.pyqtSignal(float)
    analysis_started = QtCore.pyqtSignal()

    def __init__(self):
        super().__init__()
        self.setWindowTitle('Threshold Adjustment')
        self.init_ui()

    def init_ui(self):
        layout = QtWidgets.QVBoxLayout()

        # File selection (PTU, IRF, BKG)
        file_selection_layout = QtWidgets.QFormLayout()
        self.file_ptu_input = QtWidgets.QLineEdit()
        self.file_irf_input = QtWidgets.QLineEdit()
        self.file_bkg_input = QtWidgets.QLineEdit()

        file_ptu_button = QtWidgets.QPushButton('Browse')
        file_ptu_button.clicked.connect(self.select_ptu_file)
        file_irf_button = QtWidgets.QPushButton('Browse')
        file_irf_button.clicked.connect(self.select_irf_file)
        file_bkg_button = QtWidgets.QPushButton('Browse')
        file_bkg_button.clicked.connect(self.select_bkg_file)

        file_selection_layout.addRow('Main PTU File:', self._create_file_input(self.file_ptu_input, file_ptu_button))
        file_selection_layout.addRow('IRF File:', self._create_file_input(self.file_irf_input, file_irf_button))
        file_selection_layout.addRow('Background File:', self._create_file_input(self.file_bkg_input, file_bkg_button))

        layout.addLayout(file_selection_layout)

        # Output directory selection
        output_dir_layout = QtWidgets.QHBoxLayout()
        output_dir_label = QtWidgets.QLabel('Output Directory:')
        self.output_dir_input = QtWidgets.QLineEdit()
        output_dir_button = QtWidgets.QPushButton('Browse')
        output_dir_button.clicked.connect(self.select_output_directory)
        output_dir_layout.addWidget(output_dir_label)
        output_dir_layout.addWidget(self.output_dir_input)
        output_dir_layout.addWidget(output_dir_button)
        
        layout.addLayout(output_dir_layout)

        # Chunk size input
        chunk_size_layout = QtWidgets.QHBoxLayout()
        chunk_size_label = QtWidgets.QLabel('Chunk Size:')
        self.chunk_size_input = QtWidgets.QLineEdit('30000')  # Default chunk size
        chunk_size_layout.addWidget(chunk_size_label)
        chunk_size_layout.addWidget(self.chunk_size_input)
        layout.addLayout(chunk_size_layout)

        # Load Data button
        load_data_button = QtWidgets.QPushButton('Load Data')
        load_data_button.clicked.connect(self.load_data)
        layout.addWidget(load_data_button)

        # Matplotlib canvas
        self.canvas = MatplotlibCanvas(self, width=5, height=4, dpi=100)
        layout.addWidget(self.canvas)

        # Threshold slider
        self.slider = QtWidgets.QSlider(QtCore.Qt.Orientation.Horizontal)
        self.slider.setMinimum(1)
        self.slider.setMaximum(10)
        self.slider.setValue(4)  # Default threshold multiplier
        self.slider.valueChanged.connect(self.update_plot)
        self.slider.setEnabled(False)  # Disabled until data is loaded
        layout.addWidget(self.slider)

        # Threshold label
        self.label = QtWidgets.QLabel('Threshold: 4 sigma')
        layout.addWidget(self.label)

        # Next button
        self.next_button = QtWidgets.QPushButton('Next')
        self.next_button.clicked.connect(self.start_analysis)
        self.next_button.setEnabled(False)  # Disabled until data is loaded
        layout.addWidget(self.next_button)

        self.setLayout(layout)

    def _create_file_input(self, line_edit, button):
        container = QtWidgets.QHBoxLayout()
        container.addWidget(line_edit)
        container.addWidget(button)
        widget = QtWidgets.QWidget()
        widget.setLayout(container)
        return widget

    def select_ptu_file(self):
        filename, _ = QtWidgets.QFileDialog.getOpenFileName(self, 'Select PTU File')
        if filename:
            self.file_ptu_input.setText(filename)

    def select_irf_file(self):
        filename, _ = QtWidgets.QFileDialog.getOpenFileName(self, 'Select IRF File')
        if filename:
            self.file_irf_input.setText(filename)

    def select_bkg_file(self):
        filename, _ = QtWidgets.QFileDialog.getOpenFileName(self, 'Select Background File')
        if filename:
            self.file_bkg_input.setText(filename)
            
    def select_output_directory(self):
        directory = QtWidgets.QFileDialog.getExistingDirectory(self, 'Select Output Directory')
        if directory:
            self.output_dir_input.setText(directory)

    def load_data(self):
        file_ptu = self.file_ptu_input.text()
        file_irf = self.file_irf_input.text()
        file_bkg = self.file_bkg_input.text()
        chunk_size_text = self.chunk_size_input.text()

        if not all([file_ptu, file_irf, file_bkg, chunk_size_text]):
            QtWidgets.QMessageBox.warning(self, 'Error', 'Please select all files and specify chunk size.')
            return

        try:
            self.chunk_size = int(chunk_size_text)
            if self.chunk_size <= 0:
                raise ValueError
        except ValueError:
            QtWidgets.QMessageBox.warning(self, 'Error', 'Invalid chunk size. Please enter a positive integer.')
            return

        # Load data using core functions
        try:
            self.data_ptu, self.data_irf, self.data_bkg = dat.load_ptu_files(file_ptu, file_irf, file_bkg)
        except Exception as e:
            QtWidgets.QMessageBox.warning(self, 'Error', f'Failed to load data files:\n{e}')
            return

        # Prepare data for threshold adjustment
        self.prepare_data()
        self.slider.setEnabled(True)
        self.next_button.setEnabled(True)
        self.plot_data(self.slider.value())

    def prepare_data(self):
        # Extract data
        all_macro_times = self.data_ptu.macro_times
        all_micro_times = self.data_ptu.micro_times

        # Get resolutions
        macro_res = self.data_ptu.get_header().macro_time_resolution
        micro_res = self.data_ptu.get_header().micro_time_resolution

        # Calculate interphoton arrival times
        photon_time_intervals, _ = an.interphoton_arrival_times(
            all_macro_times, all_micro_times, macro_res, micro_res
        )

        # Calculate running average
        window_size = 30
        running_avg = an.running_average(photon_time_intervals, window_size)
        self.logrunavg = np.log10(running_avg)

        # Estimate background noise
        bins_y = 141
        self.mu, self.std, _, _, _ = an.estimate_background_noise(self.logrunavg, bins_y)

    def plot_data(self, threshold_multiplier):
        threshold_value = self.mu - threshold_multiplier * self.std
        filtered_values = np.ma.masked_greater(self.logrunavg, threshold_value)

        self.canvas.axes.clear()
        self.canvas.axes.plot(self.logrunavg, label='Running Average', linestyle='None', marker='o', markersize=2)
        self.canvas.axes.plot(filtered_values, label='Threshold Values', linestyle='None', marker='.', markersize=2)
        self.canvas.axes.axhline(y=threshold_value, color='r', linestyle='--', label=f'Threshold ({threshold_multiplier}σ)')
        self.canvas.axes.set_xlabel('Photon Event #')
        self.canvas.axes.set_ylabel('log(Photon Interval Time)')
        self.canvas.axes.legend(loc='lower right')
        self.canvas.draw()

    def update_plot(self, value):
        threshold_multiplier = value
        self.label.setText(f'Threshold: {threshold_multiplier} sigma')
        self.plot_data(threshold_multiplier)
        self.threshold_changed.emit(threshold_multiplier)

    def start_analysis(self):
        # Proceed to Fit23 preview
        threshold_multiplier = self.slider.value()
        threshold_value = self.mu - threshold_multiplier * self.std
        burst_index, _ = dat.extract_greater(self.logrunavg, threshold_value)

        # Extract the entire chunk data
        from .fit23_preview import Fit23PreviewWindow

        self.fit23_window = Fit23PreviewWindow(
            self.data_ptu, self.data_irf, burst_index, self.chunk_size, self.output_directory
        )
        self.fit23_window.show()

================
File: feda_tools/gui/widgets.py
================
from PyQt6 import QtWidgets
from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg as FigureCanvas  # Compatible with PyQt6
from matplotlib.figure import Figure

class MatplotlibCanvas(FigureCanvas):
    def __init__(self, parent=None, width=5, height=4, dpi=100):
        fig = Figure(figsize=(width, height), dpi=dpi)
        self.axes = fig.add_subplot(111)
        super().__init__(fig)
        self.setParent(parent)

================
File: feda_tools/pipeline.py
================
from tqdm import tqdm
import numpy as np
import pandas as pd
import pathlib
import tttrlib

from feda_tools.core import utilities as utils
from feda_tools.core import analysis as an
from feda_tools.core import data as dat
from feda_tools.core import model
from feda_tools.core import process

def main():
    # Set file paths
    file_path = pathlib.Path('./test data/2024')
    file_source = '/Split_20230809_HighFRETDNAStd_1hr_Dani-000000.ptu'
    file_water = '/20230809_IRFddH2O_Dani_5min.ptu'
    file_buffer = '/20230809_bg_HighFRETDNAStd_30sec_Dani.ptu'
    file_ptu = str(file_path) + file_source
    file_irf = str(file_path) + file_water
    file_bkg = str(file_path) + file_buffer

    if not file_path.exists():
        raise FileNotFoundError("The provided testing path does not exist")

    # Load PTU Files
    data_ptu, data_irf, data_bkg = dat.load_ptu_files(file_ptu, file_irf, file_bkg)

    # Extract data
    all_macro_times = data_ptu.macro_times
    all_micro_times = data_ptu.micro_times
    routing_channels = data_ptu.routing_channels

    all_macro_times_irf = data_irf.macro_times
    all_micro_times_irf = data_irf.micro_times
    routing_channels_irf = data_irf.routing_channels

    all_macro_times_bkg = data_bkg.macro_times
    all_micro_times_bkg = data_bkg.micro_times
    routing_channels_bkg = data_bkg.routing_channels

    # Get resolutions
    macro_res = data_ptu.get_header().macro_time_resolution
    micro_res = data_ptu.get_header().micro_time_resolution

    # Total number of events (photons)
    total_events = len(all_macro_times)
    print("Total Events:", total_events)
    chunk_size = 30000  # Set number of indices per chunk
    num_chunks = (total_events + chunk_size - 1) // chunk_size  # Calculate the number of chunks

    # Analysis settings
    min_photon_count = 60
    bg4_micro_time_min = 0
    bg4_micro_time_max = 12499
    g_factor = 1.04
    l1_japan_corr = 0.0308
    l2_japan_corr = 0.0368
    bg4_bkg_para = 0
    bg4_bkg_perp = 0
    num_bins = 128

    # Initialize result containers
    all_bi4_bur_df = []
    all_bg4_df = []

    for chunk_idx in range(num_chunks):
        print(f"Processing chunk {chunk_idx + 1}/{num_chunks}...")

        # Define start and end indices for the current chunk
        start_idx = chunk_idx * chunk_size
        end_idx = min((chunk_idx + 1) * chunk_size, total_events)

        # Slice data for the current chunk
        macro_times_chunk = all_macro_times[start_idx:end_idx]
        micro_times_chunk = all_micro_times[start_idx:end_idx]
        routing_channels_chunk = routing_channels[start_idx:end_idx]

        # Calculate interphoton arrival times for the current chunk
        photon_time_intervals, photon_ids = an.interphoton_arrival_times(
            macro_times_chunk, micro_times_chunk, macro_res, micro_res
        )

        # Calculate running average
        window_size = 30
        running_avg = an.running_average(photon_time_intervals, window_size)
        logrunavg = np.log10(running_avg)

        # Estimate background noise
        bins = {"x": 141, "y": 141}
        bins_y = bins['y']
        mu, std, noise_mean, filtered_logrunavg, bins_logrunavg = an.estimate_background_noise(logrunavg, bins_y)

        # Define threshold value for burst extraction
        threshold_value = mu - 4 * std

        # Extract bursts for the current chunk
        burst_index, filtered_values = dat.extract_greater(logrunavg, threshold_value)

        # Setup fit23 for the current chunk
        counts_irf, _ = np.histogram(all_micro_times_irf, bins=num_bins)
        counts_irf_nb = counts_irf.copy()
        counts_irf_nb[0:3] = 0
        counts_irf_nb[10:66] = 0
        counts_irf_nb[74:128] = 0
        fit23 = model.setup_fit23(num_bins, macro_res, counts_irf_nb, g_factor, l1_japan_corr, l2_japan_corr)

        # Process bursts for the current chunk
        bi4_bur_df, bg4_df = process.process_bursts(
            burst_index, macro_times_chunk, micro_times_chunk, routing_channels_chunk, macro_res, micro_res,
            min_photon_count, bg4_micro_time_min, bg4_micro_time_max, g_factor, l1_japan_corr,
            l2_japan_corr, bg4_bkg_para, bg4_bkg_perp, fit23
        )

        # Append results for each chunk
        all_bi4_bur_df.append(bi4_bur_df)
        all_bg4_df.append(bg4_df)

    # Concatenate results from all chunks
    bi4_bur_df_combined = pd.concat(all_bi4_bur_df, ignore_index=True)
    bg4_df_combined = pd.concat(all_bg4_df, ignore_index=True)

    # Save results
    output_directory = r'./tests/burstid_selection_viz_tool'
    dat.save_results(output_directory, file_path, bi4_bur_df_combined, bg4_df_combined)

if __name__ == '__main__':
    main()

================
File: LICENSE
================
MIT License

Copyright (c) 2023 Frank-Duffy

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

================
File: README.md
================
Installation instructions:

1. Clone the repository to your local computer.
2. Create an anaconda environment exclusive to the package. Create it with Python 3.11.5 or 3.11.9.
3. Activate the environment.
4. Open terminal (on OS) or Anaconda terminal (on PC), and navigate to the root of the cloned repository.
5. Exectute >pip install -e .
6. Check that the dependencies were correctly installed by opening one of the analysis notebooks in your favorite Jupyter Notebook compatible development tool and running the import cell.

Recommendations:
- Use Anaconda-Navigator to install Spyder with Spyder-notebook and use it to interact with the analysis notebooks. There are more tools provided by Spyder than with by the basic Jupyter Server. The variable explorer offered by Spyder is very good for debugging analysis, in particular.

================
File: repopack.config.json
================
{
  "output": {
    "filePath": "repopack-output.txt",
    "style": "plain",
    "removeComments": false,
    "removeEmptyLines": false,
    "topFilesLength": 5,
    "showLineNumbers": false
  },
  "include": [],
  "ignore": {
    "useGitignore": true,
    "useDefaultPatterns": true,
    "customPatterns": []
  },
  "security": {
    "enableSecurityCheck": true
  }
}

================
File: setup.py
================
from setuptools import setup, find_packages

from feda_tools.__init__ import __version__

extra_bin = [
    'numpy',
    'pandas',
    'matplotlib',

]

extra_test = [
    'pytest>=4',
    'pytest-cov>=2',
]

extra_dev = [
    *extra_test
]

extra_ci = [
    *extra_test,
    'python-coveralls',
]

setup(
    name='feda_tools',
    version=__version__,

    url='https://github.com/SMB-Lab/feda_tools',
    author='Frank Duffy',
    author_email='fduffy0328@gmail.com',

    packages=find_packages(exclude=['tests', 'tests.*']),
    install_requires=[
        'tqdm==4.66.1',
        'numpy==1.26.2',
        'pandas==2.1.4',
        'scipy==1.11.4',
        'seaborn==0.13.0',
        'matplotlib==3.8.3',
        'pathlib==1.0.1',
        'tttrlib==0.23.9',
        'pyyaml',
    ],
    python_requires= '>=3.11.5, <=3.11.9',
    extras_require= {
        'dev' : extra_dev,
        'ci' : extra_ci,
    },
    entry_points={
    'console_scripts': [
        '2dhist=feda_tools.twodim_hist:make_2dhist',
        ],
    },
)