This will be a quick and different post, recording one of these A-HA moments we sometimes have, before I forget it. It is a random thought that’s got more than 140 characters. A tweet would not do it.

I just finished reading The Black Hole War, by Leonard Susskind (one of the most important physicists alive today). It is a great book if you are interested in modern physics. Here is an excerpt of my review on Goodreads:

(…). For non-physicists like me, this was a fantastic introduction on what we currently know about quantum gravity and its relation with other areas of science. As a bonus, it also (finally) helped me start grasping string theory, and better understand entropy, the event horizon (complementarity, information paradox) and the holographic principle.

My A-HA moment came while the book discussed the Holographic Principle. Coincidence or not, I have been studying a whole bunch of Machine Learning, and things converged. I love when (seemingly) completely different areas of science suddenly converge. It reinforces my belief that there must be something common underlying all of it (call it Grand Unified Theory, or God, or as you please).

Principal Component Analysis and Dimensionality Reduction are the Data Science (Machine Learning) topics in particular that seem to be correlated with the Holographic Principle. They are techniques of removing redundancy in data sets and extracting the minimal data necessary to represent something. It allows you to reduce the number of columns in a database without losing meaningful information, for example. The main technique can be seen as a linear algebra transformation: a projection of the N-dimensional data onto a smaller K-dimensional space.

In a sense, holograms are the same thing: a projection (encoding) of a higher dimensional space onto a smaller dimensional space (3D to 2D, for example). The Holographic Principle states that all information inside a N-dimensional space is contained into its (N-1)-dimensional boundary. For example, all the information inside a volume (3D) is contained (or described) onto its surface/area (2D).

This is fascinating. Holograms are fascinating. It could mean that not all dimensions (e.g.: rows/columns in your database) are necessary to perfectly describe any information, and removing that redundancy is precisely what Dimensionality Reduction (and PCA) tries to do. I wonder if we can use the Holographic Principle to find ways to do Dimensionality Reduction without losing any information (a loss-less compression, if you will).

Linear algebra is another fascinating aspect of all this. Projecting data and extracting meaningful information (compression) always seems to involve the calculation of Eigenvectors and Eigenvalues. Google is constantly calculating Eigenvectors that power your searches. Somehow, Eigenvectors also seem to be central to all of this.

I am sure this is not something I’m inventing or discovering and there are plenty of papers about it out there. I only happen to not have stumbled upon any of them. A quick search tells me that Holography and Dimensionality Reduction is being used in many different areas of science, including genetics and biology. If you know of any of such papers (proving or disproving my random thought), let me know in the comments.