Skip to content

Motion transfer booth for a 1 hour everybody dance now video generation using EdgeTPU and Tensorflow 2.0

License

Notifications You must be signed in to change notification settings

smellslikeml/everybody_dance_faster

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Everybody Dance Faster 💃🏽🕺

Python 3.x

Everybody Dance Faster flow

Read our blog post for the full write-up.

These instructions will show how to acquire pose estimates from a participant, train a pix2pix model, transfer source dance video, and generate a dance gif.

Prerequisites

Software

Hardware

Install

We recommend using a virtual environment to avoid any conflicts with your system's global configuration. You can install the required dependencies via pip. The EdgeTPU will require the Python API as described above.

This demo also requires that you have a Google Cloud account and have configured the python client with credentials. See this for more resources.

Data Acquisition

In the pose/ directory, you will find all of the EdgeTPU resources required to capture training images from your booth participant.

On the EdgeTPU, run:

python3 run.py

This script will generate the pose estimation overlays and raw image assets and send them to your declared Google Cloud Storage bucket.

Training

After setting up a VM with a GPU instance in your Google Cloud account, install the requirements listed above. Then you can use the train.py script to train the Pix2Pix model.

python3 train.py

Periodic checkpoints will be stored in the checkpoints/ directory. This script trains for 50 epochs to reduce training time.

Generate Dance

We've included default source video frames from the Bruno Mars - That's What I Like music video. If you are using another source video, keep the framing, perspective, and background of your source in mind for better results.

To generate a dance gif of your participant, run the following on your Google Cloud training VM instance:

python3 generate_gif.py

The asset will be stored in the results/ directory and in your declared Google Cloud Storage bucket.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE.md file for details

References

About

Motion transfer booth for a 1 hour everybody dance now video generation using EdgeTPU and Tensorflow 2.0

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages