Regarding Fractals And Non-Integral Dimensionality

Depixellation? Or hallucination?

There’s an application for neural nets called “photo upsampling” which is designed to turn a very low-resolution photo into a higher-res one.

This is an image from a recent paper demonstrating one of these algorithms, called “PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models”

It’s the neural net equivalent of shouting “enhance!” at a computer in a movie - the resulting photo is MUCH higher resolution than the original.

Could this be a privacy concern? Could someone use an algorithm like this to identify someone who’s been blurred out? Fortunately, no. The neural net can’t recover detail that doesn’t exist - all it can do is invent detail.

This becomes more obvious when you downscale a photo, give it to the neural net, and compare its upscaled version to the original.

As it turns out, there are lots of different faces that can be downscaled into that single low-res image, and the neural net’s goal is just to find one of them. Here it has found a match - why are you not satisfied?

And it’s very sensitive to the exact position of the face, as I found out in this horrifying moment below. I verified that yes, if you downscale the upscaled image on the right, you’ll get something that looks very much like the picture in the center. Stand way back from the screen and blur your eyes (basically, make your own eyes produce a lower-resolution image) and the three images below will look more and more alike. So technically the neural net did an accurate job at its task.

A tighter crop improves the image somewhat. Somewhat.

The neural net reconstructs what it’s been rewarded to see, and since it’s been trained to produce human faces, that’s what it will reconstruct. So if I were to feed it an image of a plush giraffe, for example…

Given a pixellated image of anything, it’ll invent a human face to go with it, like some kind of dystopian computer system that sees a suspect’s image everywhere. (Building an algorithm that upscales low-res images to match faces in a police database would be both a horrifying misuse of this technology and not out of character with how law enforcement currently manipulates photos to generate matches.)

However, speaking of what the neural net’s been rewarded to see - shortly after this particular neural net was released, twitter user chicken3gg posted this reconstruction:

Others then did experiments of their own, and many of them, including the authors of the original paper on the algorithm, found that the PULSE algorithm had a noticeable tendency to produce white faces, even if the input image hadn’t been of a white person. As James Vincent wrote in The Verge, “It’s a startling image that illustrates the deep-rooted biases of AI research.”

Biased AIs are a well-documented phenomenon. When its task is to copy human behavior, AI will copy everything it sees, not knowing what parts it would be better not to copy. Or it can learn a skewed version of reality from its training data. Or its task might be set up in a way that rewards - or at the least doesn’t penalize - a biased outcome. Or the very existence of the task itself (like predicting “criminality”) might be the product of bias.

In this case, the AI might have been inadvertently rewarded for reconstructing white faces if its training data (Flickr-Faces-HQ) had a large enough skew toward white faces. Or, as the authors of the PULSE paper pointed out (in response to the conversation around bias), the standard benchmark that AI researchers use for comparing their accuracy at upscaling faces is based on the CelebA HQ dataset, which is 90% white. So even if an AI did a terrible job at upscaling other faces, but an excellent job at upscaling white faces, it could still technically qualify as state-of-the-art. This is definitely a problem.

A related problem is the huge lack of diversity in the field of artificial intelligence. Even an academic project with art as its main application should not have gone all the way to publication before someone noticed that it was hugely biased. Several factors are contributing to the lack of diversity in AI, including anti-Black bias. The repercussions of this striking example of bias, and of the conversations it has sparked, are still being strongly felt in a field that’s long overdue for a reckoning.

Bonus material this week: an ongoing experiment that’s making me question not only what madlibs are, but what even are sentences. Enter your email here for a preview.

My book on AI is out, and, you can now get it any of these several ways! Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s - Boulder Bookstore

#beingacomputerprogrammingmajor : Testing out html scripts on Pokemon Go and finding out it actually works.

Computer simulations that teach themselves to walk.

Note to self: debugging

always remember to remove the Log.d(”Well fuck you too”) lines and the variables named “wtf” and “whhhyyyyyyy” before pushing anything to git

Calculating the surface area of a sphere. Found on Imgur.

A sand pendulum that creates a beautiful pattern only by its movement.

But  why does the ellipse change shape?

The pattern gets smaller because energy is not conserved (and in fact decreases) in the system. The mass in the pendulum gets smaller and the center of mass lowers as a function of time. Easy as that, an amazing pattern arises through the laws of physics.

Permutations

Figuring out how to arrange things is pretty important.

Like, if we have the letters {A,B,C}, the six ways to arrange them are: ABC ACB BAC BCA CAB CBA

And we can say more interesting things about them (e.g. Combinatorics) another great extension is when we get dynamic

Like, if we go from ABC to ACB, and back…

We can abstract away from needing to use individual letters, and say these are both “switching the 2nd and 3rd elements,” and it is the same thing both times.

Each of these switches can be more complicated than that, like going from ABCDE to EDACB is really just 1->3->4->2->5->1, and we can do it 5 times and cycle back to the start

We can also have two switches happening at once, like 1->2->3->1 and 4->5->4, and this cycles through 6 times to get to the start.

Then, let’s extend this a bit further.

First, let’s first get a better notation, and use (1 2 3) for what I called 1->2->3->1 before.

Let’s show how we can turn these permutations into a group.

Then, let’s say the identity is just keeping things the same, and call it id.

And, this repeating thing can be extended into making the group combiner: doing one permutation and then the other. For various historical reasons, the combination of permutation A and then permutation B is B·A.

This is closed, because permuting all the things and then permuting them again still keeps 1 of all the elements in an order.

Inverses exist, because you just need to put everything from the new position into the old position to reverse it.

Associativity will be left as an exercise to the reader (read: I don’t want to prove it)