Neural Fourier Transform: a method of deep equivariant representation learning

We introduce a framework of representation learning that uses group representations from underlying group action in data generation. We assume that the data consists of examples of the group action, comprising a point and its transformation under a group element, or sequences generated through the successive application of a group element. Utilizing an autoencoder architecture, our approach maps the data to a latent space in a manner that is equivariant to the group action, achieving an approximate group representation leaned from data. By applying block-diagonalization, we decompose the representation into irreducible representations, which we call the Neural Fourier Transform. This presents a generalized, data-driven approach to Fourier transform. We validate our framework across various scenarios such as image sequences, demonstrating that our derived irreducible representations effectively disentangle the underlying generative processes of the data. Theoretical results supporting our methodology are also presented.