Deepfake Cybercrime – My Horrendous Attempt

I unfortunately never managed to take a cybersecurity course until my Senior year. Though I was really looking forward to step into cybersecurity at some extent, my schedule was stacked up until I finally managed to squeeze in INFO 415 in Fall ’21 – I was excited. My journey in tech so far – from courses and educational experiences to peeks behind industry scenes through a couple of internships and partnered projects here at PODUW – had me come across the hardcore programmers, AI/ML futurists, creatives from Web Development and UI/UX design, Statisticians, and more, but yet anywhere near cybersecurity – I was excited to tap into the buzz and learn about the tool methods and communities of the surely-existing Mr. Robot world out there.

I obviously realized that a broad level course without prerequisites is not going to throw the deepest tools and concepts at me just yet; however, fast forward about a month into the course and about the biggest case study introduced was Hushpuppi, a now-arrested scammer that managed to build a massive wealth and a public influencing image through running scam operations. Not to discredit anything off of the course, it was greatly structured and Prof. Lindah Kotut was really engaging, this was simply the appropriate content in relevance to Cybersecurity. We did go on to learn more about cybersecurity through scopes like automation, blockchain, and more – and while I’m sure one course does not begin to scratch the surface of cybersecurity and cybercrime, none of what we were introduced to looked like the advanced technical space that I had imagined cybersecurity to be, nor did what seemed to lie ahead.

Spam and Phishing are still working!?

Back to what really sparked in me; spam and phishing are not only still working, but their volume has also been on the rise, verified by my inbox. And the reason half of the Nigerian royal family and two thirds of the world’s churches are in my junk inbox, is because, while seemingly mundane to me, these techniques are working somewhere for somebody. Quite large numbers of somebodies too. I mean, I completely understand how demographics of different age-groups or educational backgrounds might fall for such techniques, I just thought these narratives are way too unoriginal and easily spotted by myself or those around me to fall for.

However, I realized that mundane aspects were attributed there, in these chosen narratives and outreach methods, rather than intrinsically within the concept of spam or phishing. I mean, it made sense why these techniques serve as an extent of cybercrime foundations, they are rather easy to carry out and they attack vulnerable points within systems – people. And so if these techniques were so conceptually solid, I got to think about what the threshold would look like, for me to fall susceptible to such attempts. If it was simply about the narratives being too unoriginal or weak, how solid of a narrative would it take for my all knowing, better-than-thou-grandma, self to fall victim.

Reverse-Engineering Mundane Narratives

To tackle my now-defined question, I thought about what makes these narrative mundane in the first place, so that I can come up with an approach that avoids such falls. Disclaimer; the following series of …visuals, were put together by my all-nighter self after having put together the idea but procrastinated creating any slides for my class proposal presentation.

Level 1: classic Nigerian prince narrative – right off the bat, first thoughts would have me think, ‘pffft, I ain’t falling for this, that’s clearly not a real Nigerian prince behind that email.’

Level 2: Supposedly somehow proven to be indeed, a Nigerian prince, following thoughts would have me think something along, ‘pffft, this Nigerian prince is clearly trying to scam me and certainly does not have my best interest’.

Level 3: This very real prince somehow convinces me that he is a friend who has my interest in mind – ‘Umm I guess thanks for asking about my dog, I don’t know if I’m quite ready to get into a multi-million, totally-legal operation with you just yet.’

Level 4: What if it was indeed my friend, saying ‘hey look at me, it’s me, your friend!’ – Ahha! Surely that should do it.

I guess I reverse-reverse-engineered this last level since I arrived at it having already had deepfakes in mind. Further below is a deeper technical dive into the deepfakes and their workflow, but the technology essentially deploys trained machine learning models to produce videos of individuals saying and/or doing things that were never said and/or done. This helped me finally narrow down my inquiry, and design a plan accordingly to answer: Can deepfakes be used as a cyber-attack tool in spearphishing narratives?

The Plan

I decided to put together a hands-on plan tackling my questions, and proposed to Prof. Kotut that I write this article about my findings rather than an academic-paper deliverable. The plan put together was as follows:

Step I. Generate Deepfakes of Myself.

Step II. Share Fabricated Videos Publicly on Social Media as ‘Myself’.

Step III. Observe reactions! Would the deepfakes get called out? Would they pass by as me?

To tackle the main question at different limits, I decided to generate three different narratives to see the extent that this plan could, or fail to, push.

A low-stake level where I simply try to pass by a deepfake.

A medium-stake level where I try to pass a deepfake as myself to engage with someone.

A high-stake level where I use a deepfake to try to extract credentials. Our research question’s jackpot!

As I laid out the plan, concerns followed in two main areas: Firstly in the technical challenge, deepfakes are known to be rather costly in terms of computational resources, and having never worked with any of the packages before, I just kind of threw in the proposal hoping that by getting officially stuck with it I’ll manage one way or another. Secondly, in ethical concerns; the part of my plan where I essentially aim to deceive involuntary participants was a bit funny, incidentally being enrolled in a data-ethics course through the same quarter. To deal with the latter, I narrowed down my audience to a somewhat close social-circle, that would have me explain to them what this was for afterwards.

Deepfake Technical Workflow

Deepfaking wonderfully introduced us to snippets like her majesty, the Queen, singing Bohemian Rhapsody by the band, Queen and many other snippets shared across from social media to news outlets. These fabrications can either be produced through [third party software programs], or by training open-source models and ‘manually’ fabricating them.

For my plan, I went with the later approach using the DeepFaceLab package, which, along with the FaceSwap package, seemed to be the only two libraries with community support that would not leave me lost and alone – but rather simply, lost. The following is a simplified roadmap of what DeepFaceLab’s workflow looks like:

I. Source Compilation – Videos and images of both ‘source’ and ‘destination’ faces compiled into a workspace.

II. Frame Extraction – Videos are extracted into still images.

III. Face Extraction – Faces are located from frames, extracted, and aligned.

IV. Face Masking – Once faces are located, a model masks exact boundary outlines of faces.

V. Model Training – This is where the magic really happens, the model iterates over stills from the destination, and tries to match a predicted face from the source by trying to find the closest still match, along with some extra magic stuff.

VI. Face Merge – After the model is trained for some time, this steps runs the model to ‘predict’ faces across frames, and mash the two together.

This was a rather simplified map as about each of these steps included parameters to tweak around, and even multiple models or modes to choose from.

Our Plan in Action

Skipping over a solid period of time, of ashamedly redacted exact length, as I managed to understand and navigate the workflow and the package’s different versions and models, trying to get things set up on my M1 chip macbook*, moving over to Google Colab, and finally settling at AWS. I finally managed to get things up and running using a g4dn.xlarge EC2 instance – spoiler alert, but my previous remark around deepfake costs came into play and I ended up upping both specs and storage continuously throughout the project, and ended up paying much more than I thought I would.

The first few steps were rather smooth, I set down my phone and recorded lengthy videos of my face through ranges of angles, light settings, facial gestures and emotions, and more. I also put together a compilation of interviews featuring the Boston Celtic’s Jason Tatum for an exploratory start. I did not tweak any parameters or use any provided manual options for everything up til the model training.

I finally got the model to start training and logged out of the AWS instance to leave it doing its thing for a few hours. I returned back, excitingly to my first results, that unfortunately unexcitingly came up like this:

In the wonderful above result figure 1, we can see the model’s isolation of Jayson’s face on the left, right besides its prediction of what my face laid on top would look like on the right…

I then realized that there are forums for the sharing of pre-trained models, and while that would still require further training with my specific face sets, pre-trained models would most certainly boost progress as the model would at least grasp some distinction between a human face and a potato. I imported a model that has been pre-trained for over 400,000 iterations, what would have been over 6 months of run-time at the pace my potato-model was going at the time, saving me from a rather disappointing course final presentation.

Jump starting things with a pre-trained model, along with tuning up the spec notch a slight bit to a g4dn.4xlarge helped me arrive at my first fabricated video:

… yeah not quite there yet, but at least I had something to work from.

Fast forward through some time for more model training, playing around with some model parameters, shopping AWS specs like it was Christmas, and finalizing my narratives that have now included the signing of my brother as lead actor – the plan was ready to be deployed. Well not really based on the production quality as I would have liked to keep the model training some more, but the quarter was wrapping up and so the deadline said it needed to.

Level 1

Since I had already invested time training the model between Jayson Tatum and I’s faces, I used a clip from Tatum’s appearance on the podcast, The Old Man and the Three, with my face deepfaked atop. I slightly blurred the video and tried to play it off like I was making an appearance, and also included Instagram’s emoji slider to try to gauge some activity.

While the post did not generate tons of engagement, I would think that it, for the most part, passes the first level of simply trying to pass a deepfake by. The post received 7 replies engaging with the eye emoji slider.

Level 2

Level 2 had ‘me’ rant about how I was trying to find an excuse to keep procrastinating for finals, and an invite to hit me up, along with another interactive Instagram snippet asking about what food I should order. I had originally intended for a more organic scene rather than a produced sit-down; however, the tradeoff in quality had me stick with this option in hopes that the setting doesn’t unnaturally throw people off – you know, besides the creepily glitching horrendous face.

This had similar reception, on my end, to the previous Tatum snippet, with a few replies to the interactive Instagram feature. I assume unengaged viewers had rather different experiences here though, as it takes a rather small amount of attention to notice something being off. I have also later on asked for input whereby a couple of replies mentioned something along realizing it was fabricated in one way or another but not sure about how/why.

Level 3

On to the most explicit narrative, audaciously asking if I had ever shared any of my passwords before, along with a lifeless deepfake stare. I had the lowest expectations to get anything back from this narrative, yet received rather surprising feedback.

  • The video was sent as a direct Snapchat message to 14 friends
  • 6(!) Replied that they don’t know any passwords.
  • 4 Replies figured out that I was trying to do something fishy and laughed along.
  • One person reached out outside of Snapchat concerned about me being hacked.

Had I shared any passwords in the first place, I don’t know if these 6 people would have shared them back or not; however, I was still surprised as I had very lopsided expectations as I sent the messages out. This even managed to get past my other brother, Riyadh, who sent back a voice recording asking how he can be of help and that hopefully there was nothing too serious in any locked out accounts.

Reflection

Reflecting back on the original research question, I came out of this with an even larger apprehension about the dangerous potentials of deepfakes in cybercrime, and the overall deployment of the fast-advancing fabrication methods. That by no means resulted from any cyber-success out of the project – on the contrary, had I come across an attempt of this quality, I would have categorized it yet as mundane, and one that I would not see myself falling into. However, seeing how far I got to push things at this situational scale – a student pursuing a newly defined inquiry for a course project, having never previously worked in this space – proved just how accessible deepfaking (and many similarly-advanced technologies) have become, and equipped with the computational and social resources of a large group or organization (cybercrime or else wise) can prove very effective used one way or another.

Explore

More insights

What Are the Most Common Pet Names?

PowerNAP by: Tyler Piteo-Tarpy  Seattle’s pet licenses dataset is made up of the Seattle Animal Shelter’s collection of license issue dates, license numbers, pet species,