update

Author: mhjensen

doc/src/Projects/2026/Project1/Project1.do.txt

@@ -1,260 +1,34 @@
 TITLE: Project 1
 AUTHOR: "FYS5429/9429":"https://www.uio.no/studier/emner/matnat/fys/FYS5429/index-eng.html", Advanced machine learning and data analysis for the physical sciences, University of Oslo, Norway
-DATE: Spring semester 2025, deadline March 21
+DATE: Spring semester 2026, deadline March 20
 
 
 ======= Possible paths for project 1  =======
 
-We discuss here several paths as well as data sets for the first project (or as parts of a larger project)
-Tentative deadline March 22. We would also like to propose that people who have formed groups present eventual projects on January 30. 
+We discuss here several paths as well as data sets for the first project (or as parts of a larger project).
 
 ===== Paths for the projects =====
 
-o The computational path: Here we propose a path where you develop your own code for a convolutional or eventually recurrent neural network and apply this to data selects of your own selection. The code should be object oriented and flexible allowing for eventual extensions by including different Loss/Cost functions and other functionalities. Feel free to select data sets from those suggested below here. This code can also be extended upon by adding for example autoencoders. You can compare your own codes with implementations using TensorFlow(Keras)/PyTorch or other libraries. An alternative is to develop a code for an RNN inclusing long-term-short-memory (LSTM). This is often the starting point for large-language models and so-called transformers. Large language models are hovever not discussed during the lectures.
+* The computational path: Here we propose a path where you develop your own code for a convolutional or eventually recurrent neural network and apply this to data selects of your own selection. The code should be object oriented and flexible allowing for eventual extensions by including different Loss/Cost functions and other functionalities. Feel free toselect data sets from those suggested below here. This code can also be extended upon by adding for example autoencoders. You can compare your own codes with implementations using TensorFlow(Keras)/PyTorch or other libraries. An alternative is to develop a code for an RNN inclusing long-term-short-memory (LSTM). This is often the starting point for large-language models and so-called transformers. Large language models are hovever not discussed during the lectures.
 
-o The differential equation path: Here we propose a set of differential equations (ordinary and/or partial) to be solved first using neural networks (using either your own code or TensorFlow/Pytorch or similar libraries). Thereafter we plan to extend the set of methods for solving these equations to recurrent neural networks and autoencoders (AE). This project can also be extended into including "Physics informed machine learning":"https://github.com/maziarraissi/PINNs". Here we can discuss neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. 
+* The differential equation path: Here we propose a set of differential equations (ordinary and/or partial) to be solved first using neural networks (using either your own code or TensorFlow/Pytorch or similar libraries). Thereafter we plan to extend the set of methods for solving these equations to recurrent neural networks and autoencoders (AE). This project can also be extended into including Physics informed machine learning, see URL:"https://github.com/maziarraissi/PINNs". Here we can discuss neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. 
 
-o The application path: Here you can use the most relevant method(s) (say convolutional neural networks for images) and apply this(these) to data sets relevant for your own research.
+* The application path: Here you can use the most relevant method(s) (say convolutional neural networks for images) and apply this(these) to data sets relevant for your own research.
 
-o The Gaussian processes/Bayesian statistics path: "Kernel regression (Gaussian processes) and Bayesian statistics":"https://jenfb.github.io/bkmr/overview.html" are  popular tools in the machine learning literature. The main idea behind these approaches is to flexibly model the relationship between a large number of variables and a particular outcome (dependent variable). This can form a second part of a project where for example standard Kernel regression methods are used on a specific data set. Alternatively, participants can opt to work on a large project relevant for their own research using gaussian processes and/or Bayesian machine Learning. 
+* The Gaussian processes/Bayesian statistics path: Kernel regression (Gaussian processes) and Bayesian statistics, see URL:"https://jenfb.github.io/bkmr/overview.html", are  popular tools in the machine learning literature. The main idea behind these approaches is to flexibly model the relationship between a large number of variables and a particular outcome (dependent variable). This can form a second part of a project where for example standard Kernel regression methods are used on a specific data set. Alternatively, participants can opt to work on a large project relevant for their own research using gaussian processes and/or Bayesian machine Learning. 
 
-o Other possibilities.
+* Other possibilities.
 
 ===== Defining the data sets to analyze yourself =====
 
 You can propose own data sets that relate to your research interests or just use existing data sets from say
-o "Kaggle":"https://www.kaggle.com/datasets" 
-o The "University of California at Irvine (UCI) with its  machine learning repository":"https://archive.ics.uci.edu/ml/index.php".
-o For the differential equation problems, you can generate your own datasets, as described below.
+* "Kaggle":"https://www.kaggle.com/datasets" 
+* The "University of California at Irvine (UCI) with its  machine learning repository":"https://archive.ics.uci.edu/ml/index.php".
+* For the differential equation problems, you can generate your own datasets, as described below.
 
-o If possible, you should link the data sets with existing research and analyses thereof. Scientific articles which have used Machine Learning algorithms to analyze the data are highly welcome. Perhaps you can improve previous analyses and even publish a new article? 
+* If possible, you should link the data sets with existing research and analyses thereof. Scientific articles which have used Machine Learning algorithms to analyze the data are highly welcome. Perhaps you can improve previous analyses and even publish a new article? 
 
-o A critical assessment of the methods with ditto perspectives and recommendations is also something you need to include. For those of you familiar with report writing, the layouts discussed in for example "FYS-STK4155":"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md".
-
-
-
-===== Solving  differential equations with neural networks =====
-
-Here we describe the possible differential equations we can study
-first with neural networks and thereafter with recurrent neural
-networks and/or Autoenconders.
-
-The differential equations are given by the so-called "Lorenz attractor model":"https://encyclopediaofmath.org/index.php?title=Lorenz_attractor", and read
-
-!bt
-\[
-\frac{dx}{dt}=\sigma\left(y-x\right),
-\]
-!et
-where $\sigma =10$ is a constant
-!bt
-\[
-\frac{dy}{dt}= x\left(\rho-z\right)-y,
-\]
-!et
-with $\rho=28$ and
-!bt
-\[
-\frac{dz}{dt}=xy-\beta z
-\]
-!et
-
-with $\beta=8/3$ as our final constant.
-
-The following function is a
-simple function which sets up the solution using the ordinary
-differential library which follows _NumPy_. Here we have fixed the
-time sted $\Delta t=0.01$ and the final time $t_f=8$.
-
-The program sets $100$ random initial values and produces inputs and outputs for a neural network calculations.
-The inputs are given by the values of the array $\bm{x}$ (which contains $x,y,z$ as functions of time) for the time step $\bm{x}_t$.
-The other array defined by $\bm{x}_{t+1}$ contains the outputs (or targets) which we want the neural network to reproduce.
-
-!bc pycod
-# Common imports
-import numpy as np
-from scipy.integrate import odeint
-import matplotlib.pyplot as plt
-import os
-
-# Where to save the figures and data files
-PROJECT_ROOT_DIR = "Results"
-FIGURE_ID = "Results/FigureFiles"
-DATA_ID = "DataFiles/"
-
-if not os.path.exists(PROJECT_ROOT_DIR):
-    os.mkdir(PROJECT_ROOT_DIR)
-
-if not os.path.exists(FIGURE_ID):
-    os.makedirs(FIGURE_ID)
-
-if not os.path.exists(DATA_ID):
-    os.makedirs(DATA_ID)
-
-def image_path(fig_id):
-    return os.path.join(FIGURE_ID, fig_id)
-
-def data_path(dat_id):
-    return os.path.join(DATA_ID, dat_id)
-
-def save_fig(fig_id):
-    plt.savefig(image_path(fig_id) + ".png", format='png')
-
-
-# Selection of parameter values and setting array for time
-dt =0.01; tfinal = 8
-t = np.arange(0,tfinal+dt, dt)
-beta =8.0/3.0; rho = 28.0; sigma = 10.0
-
-# define the inputs and outputs for the neural networks
-nninput = np.zeros((100*len(t)-1,3))
-nnoutput = np.zeros((100*len(t)-1,3))
-# Define the equations to integrate
-def lorenz_derivative(xyz, t0, sigma=sigma,beta=beta,rho=rho):
-    x, y, z = xyz
-    return [sigma*(x-y), x*(rho-z)-y, x*y-beta*z]
-
-# generate 100 random initial values
-x0 = -15.0+30.0*np.random.random((100,3))
-
-# Use odeint functionality by sending in derivative function
-# Feel free to change the choice of integrator
-x_t = np.asarray([odeint(lorenz_derivative, x0_j, t) 
-                  for x0_j in x0])
-
-# define the inputs and outputs for the neural networks
-for j in range(100):
-    nninput[j*(len(t)-1):(j+1)*(len(t)-1),:] = x_t[j,:-1,:]
-    nnoutput[j*(len(t)-1):(j+1)*(len(t)-1),:] = x_t[j,1:,:]
-
-!ec
-
-The input and output variables are those we will start trying our
-network with. Your first taks is to set up a neural code (either using
-your own code or TensorFlow/PyTorch or similar libraries)) and use the
-above data to a prediction for the time evolution of Lorenz system for
-various values of the randomly chosen initial values.  Study the
-dependence of the fit as function of the architecture of the network
-(number of nodes, hidden layers and types of activation functions) and
-various regularization schemes and optimization methods like standard
-gradient descent with momentum, stochastic gradient descent with
-batches and with and without momentum and various schedulers for the
-learning rate.
-
-Feel free to change the above differential equations. As an example,
-consider the following harmonic oscillator equations solved with the
-Runge-Kutta to fourth order method. This is a one-dimensional problem
-and it produces a position $x_t$ and velocity $v_t$. You could now try
-to fit both the velocities and positions using much of the same recipe
-as for Lorenz attractor.  You will find it convenient to analyze one
-set of initial conditions first. The code is included here.
-
-
-This code is an example code that solves Newton's equations of motion
-with a given force and produces an output which in turn can be used to
-train a neural network
-
-
-!bc pycod
-# Common imports
-import numpy as np
-import pandas as pd
-from math import *
-import matplotlib.pyplot as plt
-import os
-
-# Where to save the figures and data files
-PROJECT_ROOT_DIR = "Results"
-FIGURE_ID = "Results/FigureFiles"
-DATA_ID = "DataFiles/"
-
-if not os.path.exists(PROJECT_ROOT_DIR):
-    os.mkdir(PROJECT_ROOT_DIR)
-
-if not os.path.exists(FIGURE_ID):
-    os.makedirs(FIGURE_ID)
-
-if not os.path.exists(DATA_ID):
-    os.makedirs(DATA_ID)
-
-def image_path(fig_id):
-    return os.path.join(FIGURE_ID, fig_id)
-
-def data_path(dat_id):
-    return os.path.join(DATA_ID, dat_id)
-
-def save_fig(fig_id):
-    plt.savefig(image_path(fig_id) + ".png", format='png')
-
-
-def SpringForce(v,x,t):
-#   note here that we have divided by mass and we return the acceleration
-    return  -2*gamma*v-x+Ftilde*cos(t*Omegatilde)
-
-
-def RK4(v,x,t,n,Force):
-    for i in range(n-1):
-# Setting up k1
-        k1x = DeltaT*v[i]
-        k1v = DeltaT*Force(v[i],x[i],t[i])
-# Setting up k2
-        vv = v[i]+k1v*0.5
-        xx = x[i]+k1x*0.5
-        k2x = DeltaT*vv
-        k2v = DeltaT*Force(vv,xx,t[i]+DeltaT*0.5)
-# Setting up k3
-        vv = v[i]+k2v*0.5
-        xx = x[i]+k2x*0.5
-        k3x = DeltaT*vv
-        k3v = DeltaT*Force(vv,xx,t[i]+DeltaT*0.5)
-# Setting up k4
-        vv = v[i]+k3v
-        xx = x[i]+k3x
-        k4x = DeltaT*vv
-        k4v = DeltaT*Force(vv,xx,t[i]+DeltaT)
-# Final result
-        x[i+1] = x[i]+(k1x+2*k2x+2*k3x+k4x)/6.
-        v[i+1] = v[i]+(k1v+2*k2v+2*k3v+k4v)/6.
-        t[i+1] = t[i] + DeltaT
-
-
-# Main part begins here
-
-DeltaT = 0.001
-#set up arrays 
-tfinal = 20 # in dimensionless time
-n = ceil(tfinal/DeltaT)
-# set up arrays for t, v, and x
-t = np.zeros(n)
-v = np.zeros(n)
-x = np.zeros(n)
-# Initial conditions (can change to more than one dim)
-x0 =  1.0 
-v0 = 0.0
-x[0] = x0
-v[0] = v0
-gamma = 0.2
-Omegatilde = 0.5
-Ftilde = 1.0
-# Start integrating using Euler's method
-# Note that we define the force function as a SpringForce
-RK4(v,x,t,n,SpringForce)
-
-# Plot position as function of time    
-fig, ax = plt.subplots()
-ax.set_ylabel('x[m]')
-ax.set_xlabel('t[s]')
-ax.plot(t, x)
-fig.tight_layout()
-save_fig("ForcedBlockRK4")
-plt.show()
-
-!ec
-
-
-
-
-The next step is to include recurrent neural networks. These will be discussed in connection with coming lectures.
+* A critical assessment of the methods with ditto perspectives and recommendations is also something you need to include. For those of you familiar with report writing, the layouts discussed in for example "FYS-STK4155":"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md".