3 min

Science goes School Webinar “Deepfakes”

Deepfakes, deceptively authentic-looking fake videos of people: How do they work? Alexander Adrowitzer and Alexander Buchelt gave fascinating insights

Copyright: picture on the left: Raimond Spekking; picture in the middle: deepfake created by students of the St. Pölten UAS based on: "Notbremse – Bundeskanzlerin Angela Merkel im Bundestag am 16.04.21" https://www.youtube.com/watch?v=gJbcyaw38Yo (phoenix.online) and visual material of Marlies Temper (St. Pölten UAS); picture on the right: St. Pölten UAS
Copyright: picture on the left: Raimond Spekking; picture in the middle: deepfake created by students of the St. Pölten UAS based on: "Notbremse – Bundeskanzlerin Angela Merkel im Bundestag am 16.04.21" https://www.youtube.com/watch?v=gJbcyaw38Yo (phoenix.online) and visual material of Marlies Temper (St. Pölten UAS); picture on the right: St. Pölten UAS

Jim Carrey appears in Star Trek (even in all roles!) or Austrian President Alexander Van der Bellen sings Reinhard Fendrich’s “I am from Austria”. Many of us know about those deceptively authentic-looking but faked videos – so-called “deepfakes”.

But what’s the technology behind creating these videos?

Alexander Adrowitzer, Deputy Academic Director of the bachelor degree programme Data Science and Business Analytics, and Alexander Buchelt, a student of the study programme’s 6th semester, explained the computer science behind creating deepfakes step by step in an easy and understandable manner to a 12th grade school class of the HAK/HAS Tulln and showed practical examples.

The webinar took place within the framework of the “Science goes School“ programme by the Lower Austrian Provincial Government.

 DF_1_webnews.jpg

The first thing a programme which is used to create a deepfake needs to do is to analyse the selected person’s face on photographs or video recordings and identify the face’s characteristic features through machine learning.

Such characteristic features are particularly often found in the area around the mouth. Based on these features, the programme can now tell us whether the person we are looking for is on further pictures or videos.

Now, the software knows a “formula” of how the face is built, which it can also use to reconstruct it. This “formula” is applied to the head of another person, and voila: the other person’s face has already been swapped with the “learned” face. 

The Singing Austrian President and Computing Capacity

In practical terms, finding enough visual material to learn the face is often not an obstacle to creating a deepfake these days – especially for people who receive a lot of publicity, there is more than enough video material. People with some knowledge of computer science will not find it too hard to take the necessary steps as they can use ready-made software packages.

The only thing that slows down deepfake generation is that creating convincing deepfakes on most gaming PCs on the market still takes several days from the beginning to the end. Using a gaming PC is necessary as it has a powerful graphics card built in and the deepfake calculations can be particularly efficiently performed on graphics processing units (GPUs).

If these conditions are met, convincing deepfakes can be generated relatively fast. This is how Data Science students of the St. Pölten UAS created a video of Austrian President Alexander Van der Bellen singing “I am from Austria” and gave the body of the German Chancellor Angela Merkel the face of Academic Director Marlies Temper (see cover picture).

Ethical Aspects

Next to technological processes, the webinar was dedicated to discussing ethical and security-related consequences of these and similar technologies with the pupils. After all, in addition to useful and fun applications (e.g. “rejuventing” actors in film scenes or producing scenes with deceased or unavailable actors), deepfake technology can also be used to create fake videos with the intention of deceiving or blackmailing somebody.

The students acknowledged the importance of thinking twice before publishing images or videos of themselves on the internet. After all, the more material is made available, the more convincing deepfakes can be created.

(A good example of how powerful machine learning approaches for image recognition are if there is enough training material is the web application “Quick Draw“ – just try it yourself!). The workshop leaders also revealed what other indicators often give away a deepfake these days.

QuickDraw2.jpg

Furthermore, they explained how machine learning algorithms such as image recognition methods can lead to unexpected discrimination (e.g. racial discrimination) if not properly applied.

One example is a hand dryer which has been trained to identify hands: since only visual material of people with white skin was used for the training, the device could eventually identify hands of white people but failed to start when people of colour wanted to use it.

The St. Pölten UAS places a special focus on considering these ethical aspects and consequences in applying machine learning and other big data methods. For example, the course “Ethik im Umgang mit Big Data” (Ethics in Handling Big Data) is an integral part of the curriculum for all students of the degree programme Data Science and Business Analytics.

Campaign Month #ForschenStattFaken

The webinar on the phenomenon of “deepfake” coincided with the campaign month #ForschenStattFaken of May 2021 (Reseaching not faking) by the ScienceCenter-Netzwerk, which explored ways in which knowledge transfer can help strengthen the competency of distinguishing between strong, reliable sources and weak sources and checking facts.

The St. Pölten UAS contributed several projects to the campaign month. By now, the workshop has been held with yet another school class and further dates for autumn are already planned (those who are interested are advised to visit the web page of the  Science Goes School programmes – in German). This clearly demonstrates that the Science Goes School Workshop was a great success.

You want to know more? Feel free to ask!
FH-Prof. Dr. Adrowitzer Alexander

FH-Prof. Dr. Alexander Adrowitzer

Academic Director Digital Innovation and Research (MA) Deputy Academic Director Data Science and Business Analytics (BA) Lecturer International Coordinator Department of Computer Science and Security
Mag. Dr. Czedik-Eysenberg Angelika

Mag. Dr. Angelika Czedik-Eysenberg

Science Communication and Public Engagement
Research and Knowledge Transfer
Center for Research and Cooperation