skip to content
Home

IMAGES

a network for developers and users of imaging and analysis tools
 

Other events

Subscribe to Other events feed
A weekly seminar seires with talks in all areas of applied and computational analysis, of broad interest to the mathematical community. For questions about this series or if you have suggestions, please contact: m.colbrook@damtp.cam.ac.uk
URL: http://talks.cam.ac.uk/show/index/9811
Updated: 41 min 25 sec ago

Wed 24 Apr 14:00: Feature Learning in Two-layer Neural Networks: The Effect of Data Covariance

Mon, 22/04/2024 - 09:56
Feature Learning in Two-layer Neural Networks: The Effect of Data Covariance

We study the effect of gradient-based optimization on feature learning in two-layer neural networks. We consider a setting where the number of samples is of the same order as the input dimension and show that, when the input data is isotropic, gradient descent always improves upon the initial random features model in terms of prediction risk, for a certain class of targets. Further leveraging the practical observation that data often contains additional structure, i.e., the input covariance has non-trivial alignment with the target, we prove that the class of learnable targets can be significantly extended, demonstrating a clear separation between kernel methods and two-layer neural networks in this regime.

Add to your calendar or Include in your list

Thu 25 Apr 14:00: Machine Learning and Dynamical Systems Meet in Reproducing Kernel Hilbert Spaces with Insights from Algorithmic Information Theory

Sun, 21/04/2024 - 13:14
Machine Learning and Dynamical Systems Meet in Reproducing Kernel Hilbert Spaces with Insights from Algorithmic Information Theory

Since its inception in the 19th century, through the efforts of Poincaré and Lyapunov, the theory of dynamical systems has addressed the qualitative behavior of systems as understood from models. From this perspective, modeling dynamical processes in applications demands a detailed understanding of the processes to be analyzed. This understanding leads to a model, which approximates observed reality and is often expressed by a system of ordinary/partial, underdetermined (control), deterministic/stochastic differential or difference equations. While these models are very precise for many processes, for some of the most challenging applications of dynamical systems, such as climate dynamics, brain dynamics, biological systems, or financial markets, developing such models is notably difficult. On the other hand, the field of machine learning is concerned with algorithms designed to accomplish specific tasks, whose performance improves with more data input. Applications of machine learning methods include computer vision, stock market analysis, speech recognition, recommender systems, and sentiment analysis in social media. The machine learning approach is invaluable in settings where no explicit model is formulated, but measurement data are available. This is often the case in many systems of interest, and the development of data-driven technologies is increasingly important in many applications. The intersection of the fields of dynamical systems and machine learning is largely unexplored, and the objective of this talk is to show that working in reproducing kernel Hilbert spaces offers tools for a data-based theory of nonlinear dynamical systems.

In the first part of the talk, we introduce simple methods to learn surrogate models for complex systems. We present variants of the method of Kernel Flows as simple approaches for learning the kernel that appear in the emulators we use in our work. First, we will discuss the method of parametric and nonparametric kernel flows for learning chaotic dynamical systems. We’ll also explore learning dynamical systems from irregularly sampled time series and from partial observations. We will introduce the methods of Sparse Kernel Flows and Hausdorff-metric based Kernel Flows (HMKFs) and apply them to learn 132 chaotic dynamical systems. We draw parallels between Minimum Description Length (MDL) and Regularization in Machine Learning (RML), showcasing that the method of Sparse Kernel Flows offers a natural approach to kernel learning. By considering code lengths and complexities rooted in Algorithmic Information Theory (AIT), we demonstrate that data-adaptive kernel learning can be achieved through the MDL principle, bypassing the need for cross-validation as a statistical method. Finally, we extend the method of Kernel Mode Decomposition to design kernels in view of detecting critical transitions in some fast-slow random dynamical systems.

Then, we introduce a data-based approach to estimating key quantities which arise in the study of nonlinear autonomous, control, and random dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems – with a reasonable expectation of success – once the nonlinear system has been mapped into a high or infinite dimensional Reproducing Kernel Hilbert Space. We develop computable, non-parametric estimators approximating controllability and observability energies for nonlinear systems. We apply this approach to the problem of model reduction of nonlinear control systems. It is also shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system. Finally, we show how kernel methods can be used to approximate center manifolds, propose a data-based version of the center manifold theorem, and construct Lyapunov functions for nonlinear ODEs.

Add to your calendar or Include in your list

Thu 16 May 15:00: Efficient Computation through Tuned Approximation

Mon, 15/04/2024 - 07:52
Efficient Computation through Tuned Approximation

Numerical software is being reconstructed to provide opportunities to tune dynamically the accuracy of computation to the requirements of the application, resulting in savings of memory, time, and energy. Floating point computation in science and engineering has a history of “oversolving” relative to requirements or worthiness for many models. So often are real datatypes defaulted to double precision that GPUs did not gain wide acceptance in simulation environments until they provided in hardware operations not required in their original domain of graphics. However, driven by performance or energy incentives, much of computational science is now reverting to employ lower precision arithmetic where possible. Many matrix operations considered at a blockwise level allow for lower precision and, in addition, many blocks can be approximated with low rank near equivalents. This leads to smaller memory footprint, which implies higher residency on memory hierarchies, leading in turn to less time and energy spent on data copying, which may even dwarf the savings from fewer and cheaper flops. We provide examples from several application domains, including a look at campaigns in geospatial statistics and seismic processing that earned Gordon Bell Prize finalist status in, resp., 2022 and 2023.

Add to your calendar or Include in your list

Thu 02 May 14:00: Title to be confirmed

Mon, 08/04/2024 - 19:07
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Thu 25 Apr 14:00: Title to be confirmed

Mon, 08/04/2024 - 19:05
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Thu 11 Apr 14:00: The Algorithmic Transparency Requirement

Mon, 08/04/2024 - 19:04
The Algorithmic Transparency Requirement

Deep learning still has drawbacks in terms of trustworthiness, which describes a comprehensible, fair, safe, and reliable method. To mitigate the potential risk of AI, clear obligations associated to trustworthiness have been proposed via regulatory guidelines, e.g., in the European AI Act. Therefore, a central question is to what extent trustworthy deep learning can be realized. Establishing the described properties constituting trustworthiness requires that the factors influencing an algorithmic computation can be retraced, i.e., the algorithmic implementation is transparent. We derive a mathematical framework which enables us to analyze whether a transparent implementation in a computing model is feasible. Finally, we exemplarily apply our trustworthiness framework to analyze deep learning approaches for inverse problems in digital computing models represented by Turing machines.

Add to your calendar or Include in your list

Wed 24 Apr 14:00: Title to be confirmed

Wed, 13/03/2024 - 11:37
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Wed 24 Apr 15:00: Title to be confirmed

Fri, 08/03/2024 - 11:37
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Thu 14 Mar 15:00: v Tangent Kernels

Sun, 03/03/2024 - 15:49
v Tangent Kernels

Machine learning (ML) has been profitably leveraged across a wide variety of problems in recent years. Empirical observations show that ML models from suitable functional spaces are capable of adequately efficient learning across a wide variety of disciplines. In this work (first in a planned sequence of three), we build the foundations for a generic perspective on ML model optimization and generalization dynamics. Specifically, we prove that under variants of gradient descent, “well-initialized” models solve sufficiently well-posed problems at \textit{a priori} or \textit{in situ} determinable rates. Notably, these results are obtained for a wider class of problems, loss functions, and models than the standard mean squared error and large width regime that is the focus of conventional Neural Tangent Kernel (NTK) analysis. The $\nu$ – Tangent Kernel ($\nu$TK), a functional analytic object reminiscent of the NTK , emerges naturally as a key object in our analysis and its properties function as the control for learning. We exemplify the power of our proposed perspective by showing that it applies to diverse practical problems solved using real ML models, such as classification tasks, data/regression fitting, differential equations, shape observable analysis, etc. We end with a small discussion of the numerical evidence, and the role $\nu$TKs may play in characterizing the search phase of optimization, which leads to the “well-initialized” models that are the crux of this work.

Add to your calendar or Include in your list

Thu 07 Mar 15:00: Hamiltonian simulation and optimal control

Tue, 27/02/2024 - 09:51
Hamiltonian simulation and optimal control

Hamiltonian simulation on quantum computers is one of the primary candidates for demonstration of quantum advantage. A central tool in Hamiltonian simulation is the matrix exponential. While uniform polynomial approximations (Chebyshev), best polynomial approximations, and unitary but asymptotic rational approximations (Padé) are well known and are extensively used in computational quantum mechanics, there was an important gap which has now been filled by the development of the theory and algorithms for unitary rational best approximations. This class of approximants leads to geometric numerical integrators with excellent approximation properties. In the second part of the talk I will talk about time-dependent Hamiltonians for many-body two-level systems, including a quantum algorithm for their simulation and some (classical) optimal control algorithms for quantum gate design.

Add to your calendar or Include in your list

Thu 23 May 15:00: TBA

Thu, 22/02/2024 - 15:06
TBA

Abstract not available

Add to your calendar or Include in your list

Thu 29 Feb 15:00: Efficient frequency-dependent numerical simulation of wave scattering problems

Wed, 21/02/2024 - 14:26
Efficient frequency-dependent numerical simulation of wave scattering problems

Wave propagation in homogeneous media is often modelled using integral equation methods. The boundary element method (BEM) is for integral equations what the finite element method is for partial differential equations. One difference is that BEM typically leads to dense discretization matrices. A major focus in the field has been the development of fast solvers for linear systems involving such dense matrices. Developments include the fast multipole method (FMM) and more algebraic methods based on the so-called H-matrix format. Yet, for time-harmonic wave propagation, these methods solve the original problem only for a single frequency. In this talk we focus on the frequency-sweeping problem: we aim to solve the scattering problem for a range of frequencies. We exploit the wavenumber-dependence of the dense discretization matrix for the 3D Helmholtz equation and demonstrate a memory-compact representation of all integral operators involved which is valid for a continuous range of frequencies, yet comes with a cost of a only small number of single frequency simulations. This is joined work at KU Leuven with Simon Dirckx, Kobe Bruyninckx and Karl Meerbergen.

Add to your calendar or Include in your list

Mon 19 Feb 14:00: SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning

Mon, 19/02/2024 - 21:12
SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning

Deep Reinforcement Learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak reactor and minimizing the drag force exerted on an object in a fluid flow. However, these algorithms require many training examples and can become prohibitively expensive for many applications. In addition, the reliance on deep neural networks results in an uninterpretable, black-box policy that may be too computationally challenging to use with certain embedded systems. Recent advances in sparse dictionary learning, such as the Sparse Identification of Nonlinear Dynamics (SINDy), have shown to be a promising method for creating efficient and interpretable data-driven models in the low-data regime. In this work, we extend ideas from the SIN Dy literature to introduce a unifying framework for combining sparse dictionary learning and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy. We demonstrate the effectiveness of our approaches on benchmark control environments and challenging fluids problems, achieving comparable performance to state-of-the-art DRL algorithms using significantly fewer interactions in the environment and an interpretable control policy orders of magnitude smaller than a deep neural network policy.

Add to your calendar or Include in your list

Wed 14 Feb 14:00: SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning

Wed, 14/02/2024 - 13:35
SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning

Deep Reinforcement Learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as stabilizing the magnetohydrodynamics of a tokamak reactor and minimizing the drag force exerted on an object in a fluid flow. However, these algorithms require many training examples and can become prohibitively expensive for many applications. In addition, the reliance on deep neural networks results in an uninterpretable, black-box policy that may be too computationally challenging to use with certain embedded systems. Recent advances in sparse dictionary learning, such as the Sparse Identification of Nonlinear Dynamics (SINDy), have shown to be a promising method for creating efficient and interpretable data-driven models in the low-data regime. In this work, we extend ideas from the SIN Dy literature to introduce a unifying framework for combining sparse dictionary learning and DRL to create efficient, interpretable, and trustworthy representations of the dynamics model, reward function, and control policy. We demonstrate the effectiveness of our approaches on benchmark control environments and challenging fluids problems, achieving comparable performance to state-of-the-art DRL algorithms using significantly fewer interactions in the environment and an interpretable control policy orders of magnitude smaller than a deep neural network policy.

Add to your calendar or Include in your list

Thu 22 Feb 15:00: Computing lower eigenvalues on rough domains

Tue, 06/02/2024 - 22:54
Computing lower eigenvalues on rough domains

In this talk I will describe a strategy for finding sharp upper and lower numerical bounds of the Poincare constant on a class of planar domains with piecewise self-similar boundary. The approach is developed in [A] and it consists of four main blocks: 1) tight inner-outer shape interpolation, 2) conformal mapping of the approximate polygonal regions, 3) grad-div system formulation of the spectral problem and 4) computation of the eigenvalue bounds. After describing the method, justifying its validity and reporting on general convergence estimates, I will show concrete evidence of its effectiveness on the Koch snowflake. I will conclude the talk by discussing potential applications to other linear operators on rough regions. This research has been conducted jointly with Lehel Banjai (Heriot-Watt University).

[A] J. Fractal Geometry 8 (2021) No. 2, pp. 153-188

Add to your calendar or Include in your list

Thu 15 Feb 15:00: Adaptive Intrusive Methods for Forward UQ in PDEs

Sun, 28/01/2024 - 17:54
Adaptive Intrusive Methods for Forward UQ in PDEs

In this talk we discuss a so-called intrusive approach for the forward propagation of uncertainty in PDEs with uncertain coefficients. Specifically, we focus on stochastic Galerkin finite element methods (SGFEMs). Multilevel variants of such methods provide polynomial-based surrogates with spatial coefficients that reside in potentially different finite element spaces. For elliptic PDEs with diffusion coefficients represented as affine functions of countably infinitely many parameters, well established theoretical results state that such methods can achieve rates of convergence independent of the number of input parameters, thereby breaking the curse of dimensionality. Moreover, for nice enough test problems, it is even possible to prove convergence rates afforded to the chosen finite element method for the associated deterministic PDE . However, achieving these rates in practice using automated computational algorithms remains highly challenging, and non-intrusive multilevel sampling methods are often preferred for their ease of use. We discuss an adaptive framework that is driven by a classical hierarchical a posteriori error estimation strategy — modified for the more challenging parametric PDE setting — and present numerical results.

Add to your calendar or Include in your list

Thu 01 Feb 15:00: What happens when you chop an equation?

Sat, 27/01/2024 - 11:05
What happens when you chop an equation?

This talk will discuss a tricky business: truncating a differential equation to produce finite solutions. A truncation scheme is often built directly into the steps needed to create a numerical system. E.g., finite differences replace exact differential operators with more manageable shadows, sweeping the exact approach off the stage.

In contrast, this talk will discuss the “tau method” which adds an explicit parameterised perturbation to an original equation. By design, the correction calls into existence an exact (finite polynomial) solution to the updated analytic system. The hope is that the correction comes out minuscule after comparing it with a hypothetical exact solution. The tau method has worked splendidly in practice, starting with Lanczos’s original 1938 paper outlining the philosophy. However, why the scheme works so well (and when it fails) remains comparably obscure. While addressing the theory behind the Tau method, this talk will answer at least one conceptual question: Where does an infinite amount of spectrum go when transitioning from a continuous differential equation to an exact finite matrix representation?

Add to your calendar or Include in your list

Thu 25 Jan 15:00: The future of governing equations

Thu, 25/01/2024 - 14:11
The future of governing equations

A major challenge in the study of dynamical systems is that of model discovery: turning data into reduced order models that are not just predictive, but provide insight into the nature of the underlying dynamical system that generated the data. We introduce a number of data-driven strategies for discovering nonlinear multiscale dynamical systems and their embeddings from data. We consider two canonical cases: (i) systems for which we have full measurements of the governing variables, and (ii) systems for which we have incomplete measurements. For systems with full state measurements, we show that the recent sparse identification of nonlinear dynamical systems (SINDy) method can discover governing equations with relatively little data and introduce a sampling method that allows SIN Dy to scale efficiently to problems with multiple time scales, noise and parametric dependencies. For systems with incomplete observations, we show that the Hankel alternative view of Koopman (HAVOK) method, based on time-delay embedding coordinates and the dynamic mode decomposition, can be used to obtain a linear models and Koopman invariant measurement systems that nearly perfectly captures the dynamics of nonlinear quasiperiodic systems. Neural networks are used in targeted ways to aid in the model reduction process. Together, these approaches provide a suite of mathematical strategies for reducing the data required to discover and model nonlinear multiscale systems.

Add to your calendar or Include in your list

Thu 25 Jan 15:00: The future of governing equations

Sun, 21/01/2024 - 12:28
The future of governing equations

A major challenge in the study of dynamical systems is that of model discovery: turning data into reduced order models that are not just predictive, but provide insight into the nature of the underlying dynamical system that generated the data. We introduce a number of data-driven strategies for discovering nonlinear multiscale dynamical systems and their embeddings from data. We consider two canonical cases: (i) systems for which we have full measurements of the governing variables, and (ii) systems for which we have incomplete measurements. For systems with full state measurements, we show that the recent sparse identification of nonlinear dynamical systems (SINDy) method can discover governing equations with relatively little data and introduce a sampling method that allows SIN Dy to scale efficiently to problems with multiple time scales, noise and parametric dependencies. For systems with incomplete observations, we show that the Hankel alternative view of Koopman (HAVOK) method, based on time-delay embedding coordinates and the dynamic mode decomposition, can be used to obtain a linear models and Koopman invariant measurement systems that nearly perfectly captures the dynamics of nonlinear quasiperiodic systems. Neural networks are used in targeted ways to aid in the model reduction process. Together, these approaches provide a suite of mathematical strategies for reducing the data required to discover and model nonlinear multiscale systems.

Add to your calendar or Include in your list

Thu 18 Jan 15:00: Computing the Spectra and Pseudospectra of Band-Dominated and Random Operators

Thu, 11/01/2024 - 09:08
Computing the Spectra and Pseudospectra of Band-Dominated and Random Operators

I will give an overview of my work over the last 15 years, with collaborators including Marko Lindner (TU Hamburg), Ratchanikorn Chonchaiya (King Mongkut’s University of Technology, Bangkok), Raffael Hagger (Kiel), and Brian Davies (KCL), on computing the spectra and pseudospectra of banded and band-dominated operators. This will include describing algorithms that, given appropriate inputs, can produce a convergent sequence of approximations to the spectrum of an arbitrary band-dominated operator, with the property that each member of the sequence can be computed in finitely many arithmetical operations. We give a concrete implementation of the algorithm for operators that are pseudoergodic in the sense of Davies (Commun. Math. Phys. 2001) and illustrate this algorithm by spectral computations for the beautiful Feinberg-Zee random hopping matrix. Details can be found at https://arxiv.org/abs/2401.03984

Add to your calendar or Include in your list

Contact us

Study at Cambridge

About the University

Research at Cambridge