skip to content

IMAGES

a network for developers and users of imaging and analysis tools
 
Subscribe to Other events feed
A weekly seminar seires with talks in all areas of applied and computational analysis, of broad interest to the mathematical community. For questions about this series or if you have suggestions, please contact: m.colbrook@damtp.cam.ac.uk
Updated: 29 min 16 sec ago

Tue 28 May 15:00: Spectral inclusions and approximations of finite and infinite banded matrices

Tue, 28/05/2024 - 10:00
Spectral inclusions and approximations of finite and infinite banded matrices

We derive inclusion sets and approximations to spectrum and pseudospectrum of banded, in general non-normal, matrices of finite or infinite size. In the infinite case (bi- or semi-infinite), the matrix acts as a bounded linear operator on the corresponding l^2 space, and we moreover bound and approximate its essential spectrum.

Our inclusion sets come as unions of pseudospectra of certain submatrices of chosen size. Via this choice, we can balance accuracy against numerical cost. The philosophy is to split one global spectral problem into several local problems of moderate size.

Add to your calendar or Include in your list

Thu 13 Jun 15:00: Finite Element Exterior Calculus for Hamiltonian PDEs

Fri, 24/05/2024 - 18:43
Finite Element Exterior Calculus for Hamiltonian PDEs

We consider the application of finite element exterior calculus (FEEC) methods to a class of canonical Hamiltonian PDE systems involving differential forms. Solutions to these systems satisfy a local multisymplectic conservation law, which generalizes the more familiar symplectic conservation law for Hamiltonian systems of ODEs, and which is connected with physically-important reciprocity phenomena, such as Lorentz reciprocity in electromagnetics. We characterize hybrid FEEC methods whose numerical traces satisfy a version of the multisymplectic conservation law, and we apply this characterization to several specific classes of FEEC methods, including conforming Arnold–Falk–Winther-type methods and various hybridizable discontinuous Galerkin (HDG) methods. Interestingly, the HDG -type and other nonconforming methods are shown, in general, to be multisymplectic in a stronger sense than the conforming FEEC methods. This substantially generalizes previous work of McLachlan and Stern [Found. Comput. Math., 20 (2020), pp. 35–69] on the more restricted class of canonical Hamiltonian PDEs in the de Donder–Weyl grad-div form.

Add to your calendar or Include in your list

Thu 13 Jun 15:00: Finite Element Exterior Calculus for Hamiltonian PDEs

Fri, 24/05/2024 - 08:01
Finite Element Exterior Calculus for Hamiltonian PDEs

We consider the application of finite element exterior calculus (FEEC) methods to a class of canonical Hamiltonian PDE systems involving differential forms. Solutions to these systems satisfy a local multisymplectic conservation law, which generalizes the more familiar symplectic conservation law for Hamiltonian systems of ODEs, and which is connected with physically-important reciprocity phenomena, such as Lorentz reciprocity in electromagnetics. We characterize hybrid FEEC methods whose numerical traces satisfy a version of the multisymplectic conservation law, and we apply this characterization to several specific classes of FEEC methods, including conforming Arnold–Falk–Winther-type methods and various hybridizable discontinuous Galerkin (HDG) methods. Interestingly, the HDG -type and other nonconforming methods are shown, in general, to be multisymplectic in a stronger sense than the conforming FEEC methods. This substantially generalizes previous work of McLachlan and Stern [Found. Comput. Math., 20 (2020), pp. 35–69] on the more restricted class of canonical Hamiltonian PDEs in the de Donder–Weyl grad-div form.

Add to your calendar or Include in your list

Thu 06 Jun 15:00: Singular flows, zeroth order pseudodifferential operators and spectra

Wed, 22/05/2024 - 13:59
Singular flows, zeroth order pseudodifferential operators and spectra

The propagation of internal gravity waves in stratified media (such as those found in ocean basins and lakes) leads to the development of attractors. These structures accumulate much of the wave energy and can make the fluid flow highly singular. These questions have been the subject of fascinating recent analytical developments by de Verdiere & Saint-Raymond, and Zworski and co-workers, who examine a simplified model which retains many of the important features. These are related to a certain zeroth-order pseudodifferential operator.

In this talk, we first review the physical phenomenon, and the (highly simplified) model evolution problem. We next describe a high-accuracy computational method to solve the evolution problem, whose long-term behaviour is known to be non-square-integrable. Then, we use similar tools to discretize the corresponding eigenvalue problem. Since the eigenvalues are embedded in a continuous spectrum, their computation is based on viscous approximations. We also study the long-term evolution of the dynamics of the system. This is joint work with Javier Almonacid.

Add to your calendar or Include in your list

Thu 06 Jun 15:00: Singular flows, zeroth order pseudodifferential operators and spectra

Tue, 21/05/2024 - 21:46
Singular flows, zeroth order pseudodifferential operators and spectra

The propagation of internal gravity waves in stratified media (such as those found in ocean basins and lakes) leads to the development of attractors. These structures accumulate much of the wave energy and can make the fluid flow highly singular. These questions have been the subject of fascinating recent analytical developments by de Verdiere & Saint-Raymond, and Zworski and co-workers, who examine a simplified model which retains many of the important features. These are related to a certain zeroth-order pseudodifferential operator.

In this talk, we first review the physical phenomenon, and the (highly simplified) model evolution problem. We next describe a high-accuracy computational method to solve the evolution problem, whose long-term behaviour is known to be non-square-integrable. Then, we use similar tools to discretize the corresponding eigenvalue problem. Since the eigenvalues are embedded in a continuous spectrum, their computation is based on viscous approximations. We also study the long-term evolution of the dynamics of the system. This is joint work with Javier Almonacid.

Add to your calendar or Include in your list

Thu 16 May 16:00: Efficient Computation through Tuned Approximation

Mon, 13/05/2024 - 18:22
Efficient Computation through Tuned Approximation

Numerical software is being reconstructed to provide opportunities to tune dynamically the accuracy of computation to the requirements of the application, resulting in savings of memory, time, and energy. Floating point computation in science and engineering has a history of “oversolving” relative to requirements or worthiness for many models. So often are real datatypes defaulted to double precision that GPUs did not gain wide acceptance in simulation environments until they provided in hardware operations not required in their original domain of graphics. However, driven by performance or energy incentives, much of computational science is now reverting to employ lower precision arithmetic where possible. Many matrix operations considered at a blockwise level allow for lower precision and, in addition, many blocks can be approximated with low rank near equivalents. This leads to smaller memory footprint, which implies higher residency on memory hierarchies, leading in turn to less time and energy spent on data copying, which may even dwarf the savings from fewer and cheaper flops. We provide examples from several application domains, including a look at campaigns in geospatial statistics and seismic processing that earned Gordon Bell Prize finalist status in, resp., 2022 and 2023.

Add to your calendar or Include in your list

Thu 23 May 15:00: A Converging Discrete Geometric Calculus on the Space of Curves

Tue, 07/05/2024 - 13:07
A Converging Discrete Geometric Calculus on the Space of Curves

The talk will take into account the space of curves as a Riemannian manifold with a metric, measuring the squared $L^2$ norm of arc length derivatives of curve variations. Based on a suitable time discretization it will be described how to interpolate pairs of curves, smoothly extrapolate paths in this space, and how to approximate the associated covariant derivative as well as the curvature tensor. The convergence of the discrete calculus to the corresponding continuous calculus will be demonstrated.

Add to your calendar or Include in your list

Thu 02 May 15:00: Greedy-LASSO, Greedy-Net: Generalization and unrolling of greedy sparse recovery algorithms

Thu, 02/05/2024 - 10:17
Greedy-LASSO, Greedy-Net: Generalization and unrolling of greedy sparse recovery algorithms

Sparse recovery generally aims at reconstructing a sparse vector, given linear measurements performed via a mixture (or sensing) matrix, typically underdetermined. Greedy (and thresholding) sparse recovery algorithms have known to serve well as a suitable alternative for convex optimization techniques, in particular in low sparsity regimes. In this talk, I take orthogonal matching pursuit (OMP) as an example, and establish a connection between OMP and convex optimization decoders in one side and neural networks on the other side. To achieve the former, we adopt a loss function-based perspective and propose a framework based on OMP that leads to greedy algorithms for a large class of loss functions including the well-known (weighted-)LASSO family, with explicit formulas for the choice of the ``greedy selection criterion”. We show numerically that these greedy algorithms inherit properties of their ancestor convex decoder. In the second part of the talk, we leverage ``softsoring”, to resolve the non-differentiability issue of OMP due to (arg)sorting, in order to derive a differentiable version of OMP that we call ``Soft-OMP”, which we demonstrate numerically and theoretically that is a good approximation for OMP . We then unroll iterations of OMP onto layers of a neural network with weights as semantic trainable parameters that capture the structure within the data. Doing so, we also connect our approach to learning weights in weighted sparse recovery. I will conclude the talk by presenting implications of our framework for other greedy algorithms such as CoSaMP and IHT , and highlight some open problems. This is joint work with Simone Brugiapaglia (Concordia University) and Matthew Colbrook (University of Cambridge).

Add to your calendar or Include in your list

Thu 02 May 14:00: Greedy-LASSO, Greedy-Net: Generalization and unrolling of greedy sparse recovery algorithms

Wed, 01/05/2024 - 14:11
Greedy-LASSO, Greedy-Net: Generalization and unrolling of greedy sparse recovery algorithms

Sparse recovery generally aims at reconstructing a sparse vector, given linear measurements performed via a mixture (or sensing) matrix, typically underdetermined. Greedy (and thresholding) sparse recovery algorithms have known to serve well as a suitable alternative for convex optimization techniques, in particular in low sparsity regimes. In this talk, I take orthogonal matching pursuit (OMP) as an example, and establish a connection between OMP and convex optimization decoders in one side and neural networks on the other side. To achieve the former, we adopt a loss function-based perspective and propose a framework based on OMP that leads to greedy algorithms for a large class of loss functions including the well-known (weighted-)LASSO family, with explicit formulas for the choice of the ``greedy selection criterion”. We show numerically that these greedy algorithms inherit properties of their ancestor convex decoder. In the second part of the talk, we leverage ``softsoring”, to resolve the non-differentiability issue of OMP due to (arg)sorting, in order to derive a differentiable version of OMP that we call ``Soft-OMP”, which we demonstrate numerically and theoretically that is a good approximation for OMP . We then unroll iterations of OMP onto layers of a neural network with weights as semantic trainable parameters that capture the structure within the data. Doing so, we also connect our approach to learning weights in weighted sparse recovery. I will conclude the talk by presenting implications of our framework for other greedy algorithms such as CoSaMP and IHT , and highlight some open problems. This is joint work with Simone Brugiapaglia (Concordia University) and Matthew Colbrook (University of Cambridge).

Add to your calendar or Include in your list

Thu 25 Apr 15:00: Machine Learning and Dynamical Systems Meet in Reproducing Kernel Hilbert Spaces with Insights from Algorithmic Information Theory

Thu, 25/04/2024 - 10:37
Machine Learning and Dynamical Systems Meet in Reproducing Kernel Hilbert Spaces with Insights from Algorithmic Information Theory

Since its inception in the 19th century, through the efforts of Poincaré and Lyapunov, the theory of dynamical systems has addressed the qualitative behavior of systems as understood from models. From this perspective, modeling dynamical processes in applications demands a detailed understanding of the processes to be analyzed. This understanding leads to a model, which approximates observed reality and is often expressed by a system of ordinary/partial, underdetermined (control), deterministic/stochastic differential or difference equations. While these models are very precise for many processes, for some of the most challenging applications of dynamical systems, such as climate dynamics, brain dynamics, biological systems, or financial markets, developing such models is notably difficult. On the other hand, the field of machine learning is concerned with algorithms designed to accomplish specific tasks, whose performance improves with more data input. Applications of machine learning methods include computer vision, stock market analysis, speech recognition, recommender systems, and sentiment analysis in social media. The machine learning approach is invaluable in settings where no explicit model is formulated, but measurement data are available. This is often the case in many systems of interest, and the development of data-driven technologies is increasingly important in many applications. The intersection of the fields of dynamical systems and machine learning is largely unexplored, and the objective of this talk is to show that working in reproducing kernel Hilbert spaces offers tools for a data-based theory of nonlinear dynamical systems.

In the first part of the talk, we introduce simple methods to learn surrogate models for complex systems. We present variants of the method of Kernel Flows as simple approaches for learning the kernel that appear in the emulators we use in our work. First, we will discuss the method of parametric and nonparametric kernel flows for learning chaotic dynamical systems. We’ll also explore learning dynamical systems from irregularly sampled time series and from partial observations. We will introduce the methods of Sparse Kernel Flows and Hausdorff-metric based Kernel Flows (HMKFs) and apply them to learn 132 chaotic dynamical systems. We draw parallels between Minimum Description Length (MDL) and Regularization in Machine Learning (RML), showcasing that the method of Sparse Kernel Flows offers a natural approach to kernel learning. By considering code lengths and complexities rooted in Algorithmic Information Theory (AIT), we demonstrate that data-adaptive kernel learning can be achieved through the MDL principle, bypassing the need for cross-validation as a statistical method. Finally, we extend the method of Kernel Mode Decomposition to design kernels in view of detecting critical transitions in some fast-slow random dynamical systems.

Then, we introduce a data-based approach to estimating key quantities which arise in the study of nonlinear autonomous, control, and random dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems – with a reasonable expectation of success – once the nonlinear system has been mapped into a high or infinite dimensional Reproducing Kernel Hilbert Space. We develop computable, non-parametric estimators approximating controllability and observability energies for nonlinear systems. We apply this approach to the problem of model reduction of nonlinear control systems. It is also shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system. Finally, we show how kernel methods can be used to approximate center manifolds, propose a data-based version of the center manifold theorem, and construct Lyapunov functions for nonlinear ODEs.

Add to your calendar or Include in your list

Wed 24 Apr 14:00: Feature Learning in Two-layer Neural Networks: The Effect of Data Covariance

Mon, 22/04/2024 - 09:56
Feature Learning in Two-layer Neural Networks: The Effect of Data Covariance

We study the effect of gradient-based optimization on feature learning in two-layer neural networks. We consider a setting where the number of samples is of the same order as the input dimension and show that, when the input data is isotropic, gradient descent always improves upon the initial random features model in terms of prediction risk, for a certain class of targets. Further leveraging the practical observation that data often contains additional structure, i.e., the input covariance has non-trivial alignment with the target, we prove that the class of learnable targets can be significantly extended, demonstrating a clear separation between kernel methods and two-layer neural networks in this regime.

Add to your calendar or Include in your list

Thu 25 Apr 14:00: Machine Learning and Dynamical Systems Meet in Reproducing Kernel Hilbert Spaces with Insights from Algorithmic Information Theory

Sun, 21/04/2024 - 13:14
Machine Learning and Dynamical Systems Meet in Reproducing Kernel Hilbert Spaces with Insights from Algorithmic Information Theory

Since its inception in the 19th century, through the efforts of Poincaré and Lyapunov, the theory of dynamical systems has addressed the qualitative behavior of systems as understood from models. From this perspective, modeling dynamical processes in applications demands a detailed understanding of the processes to be analyzed. This understanding leads to a model, which approximates observed reality and is often expressed by a system of ordinary/partial, underdetermined (control), deterministic/stochastic differential or difference equations. While these models are very precise for many processes, for some of the most challenging applications of dynamical systems, such as climate dynamics, brain dynamics, biological systems, or financial markets, developing such models is notably difficult. On the other hand, the field of machine learning is concerned with algorithms designed to accomplish specific tasks, whose performance improves with more data input. Applications of machine learning methods include computer vision, stock market analysis, speech recognition, recommender systems, and sentiment analysis in social media. The machine learning approach is invaluable in settings where no explicit model is formulated, but measurement data are available. This is often the case in many systems of interest, and the development of data-driven technologies is increasingly important in many applications. The intersection of the fields of dynamical systems and machine learning is largely unexplored, and the objective of this talk is to show that working in reproducing kernel Hilbert spaces offers tools for a data-based theory of nonlinear dynamical systems.

In the first part of the talk, we introduce simple methods to learn surrogate models for complex systems. We present variants of the method of Kernel Flows as simple approaches for learning the kernel that appear in the emulators we use in our work. First, we will discuss the method of parametric and nonparametric kernel flows for learning chaotic dynamical systems. We’ll also explore learning dynamical systems from irregularly sampled time series and from partial observations. We will introduce the methods of Sparse Kernel Flows and Hausdorff-metric based Kernel Flows (HMKFs) and apply them to learn 132 chaotic dynamical systems. We draw parallels between Minimum Description Length (MDL) and Regularization in Machine Learning (RML), showcasing that the method of Sparse Kernel Flows offers a natural approach to kernel learning. By considering code lengths and complexities rooted in Algorithmic Information Theory (AIT), we demonstrate that data-adaptive kernel learning can be achieved through the MDL principle, bypassing the need for cross-validation as a statistical method. Finally, we extend the method of Kernel Mode Decomposition to design kernels in view of detecting critical transitions in some fast-slow random dynamical systems.

Then, we introduce a data-based approach to estimating key quantities which arise in the study of nonlinear autonomous, control, and random dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems – with a reasonable expectation of success – once the nonlinear system has been mapped into a high or infinite dimensional Reproducing Kernel Hilbert Space. We develop computable, non-parametric estimators approximating controllability and observability energies for nonlinear systems. We apply this approach to the problem of model reduction of nonlinear control systems. It is also shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system. Finally, we show how kernel methods can be used to approximate center manifolds, propose a data-based version of the center manifold theorem, and construct Lyapunov functions for nonlinear ODEs.

Add to your calendar or Include in your list

Thu 16 May 15:00: Efficient Computation through Tuned Approximation

Mon, 15/04/2024 - 07:52
Efficient Computation through Tuned Approximation

Numerical software is being reconstructed to provide opportunities to tune dynamically the accuracy of computation to the requirements of the application, resulting in savings of memory, time, and energy. Floating point computation in science and engineering has a history of “oversolving” relative to requirements or worthiness for many models. So often are real datatypes defaulted to double precision that GPUs did not gain wide acceptance in simulation environments until they provided in hardware operations not required in their original domain of graphics. However, driven by performance or energy incentives, much of computational science is now reverting to employ lower precision arithmetic where possible. Many matrix operations considered at a blockwise level allow for lower precision and, in addition, many blocks can be approximated with low rank near equivalents. This leads to smaller memory footprint, which implies higher residency on memory hierarchies, leading in turn to less time and energy spent on data copying, which may even dwarf the savings from fewer and cheaper flops. We provide examples from several application domains, including a look at campaigns in geospatial statistics and seismic processing that earned Gordon Bell Prize finalist status in, resp., 2022 and 2023.

Add to your calendar or Include in your list

Thu 02 May 14:00: Title to be confirmed

Mon, 08/04/2024 - 19:07
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Thu 25 Apr 14:00: Title to be confirmed

Mon, 08/04/2024 - 19:05
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Thu 11 Apr 14:00: The Algorithmic Transparency Requirement

Mon, 08/04/2024 - 19:04
The Algorithmic Transparency Requirement

Deep learning still has drawbacks in terms of trustworthiness, which describes a comprehensible, fair, safe, and reliable method. To mitigate the potential risk of AI, clear obligations associated to trustworthiness have been proposed via regulatory guidelines, e.g., in the European AI Act. Therefore, a central question is to what extent trustworthy deep learning can be realized. Establishing the described properties constituting trustworthiness requires that the factors influencing an algorithmic computation can be retraced, i.e., the algorithmic implementation is transparent. We derive a mathematical framework which enables us to analyze whether a transparent implementation in a computing model is feasible. Finally, we exemplarily apply our trustworthiness framework to analyze deep learning approaches for inverse problems in digital computing models represented by Turing machines.

Add to your calendar or Include in your list

Wed 24 Apr 14:00: Title to be confirmed

Wed, 13/03/2024 - 11:37
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Wed 24 Apr 15:00: Title to be confirmed

Fri, 08/03/2024 - 11:37
Title to be confirmed

Abstract not available

Add to your calendar or Include in your list

Thu 14 Mar 15:00: v Tangent Kernels

Sun, 03/03/2024 - 15:49
v Tangent Kernels

Machine learning (ML) has been profitably leveraged across a wide variety of problems in recent years. Empirical observations show that ML models from suitable functional spaces are capable of adequately efficient learning across a wide variety of disciplines. In this work (first in a planned sequence of three), we build the foundations for a generic perspective on ML model optimization and generalization dynamics. Specifically, we prove that under variants of gradient descent, “well-initialized” models solve sufficiently well-posed problems at \textit{a priori} or \textit{in situ} determinable rates. Notably, these results are obtained for a wider class of problems, loss functions, and models than the standard mean squared error and large width regime that is the focus of conventional Neural Tangent Kernel (NTK) analysis. The $\nu$ – Tangent Kernel ($\nu$TK), a functional analytic object reminiscent of the NTK , emerges naturally as a key object in our analysis and its properties function as the control for learning. We exemplify the power of our proposed perspective by showing that it applies to diverse practical problems solved using real ML models, such as classification tasks, data/regression fitting, differential equations, shape observable analysis, etc. We end with a small discussion of the numerical evidence, and the role $\nu$TKs may play in characterizing the search phase of optimization, which leads to the “well-initialized” models that are the crux of this work.

Add to your calendar or Include in your list