CCI_ADV_Catchup

🥳

1.What I have done

Read essays about Pix2Code cases and try some… I gave up due to the huge amount of machine learning data for almost two months without certain positive feedback. Process

2.What I will do

Try model training and style transfer.

Background：There are multiple ways of performing style transfer on videos, the simplest of which is to repeatedly loop the still image style transfer process on each successive frame in a video. Other approaches/improvements on this method deal with optimizing smoothness of the output video from frame to frame. We chose to go with the simplest method of style transfer for videos by continuously performing image to image style transfer on each frame of the input video.

My goal: implementing style transfer on real-time video

First, train the neural network on styles we are interested in. https://www.tensorflow.org/tutorials/generative/style_transfer?hl=zh-cn

Next, take input from the webcam in the form of a streaming video.

Then, sample the video stream to acquire still images.

Finally, stylize and display the webcam images.

The end result may be a live reflection of what is seen in the webcam with a style applied on top, further, embed it in a web page.

The difficulty/challenge: Train the model

Some Results：

Todo：

1.add manual adjustment for the effect

2.Use learner. js to add a quick training effect, such as when people’s action or expression is captured by the camera, we can adopt a corresponding style transfer, a timely matching and dynamic adjustment.

Useful Resource:

real-time-Style-Transfer

real-time-style-transfer

real-time-style-transfer

Real-Time Style Transfer_ Artistic stylization on real-time video

Fast Neural Style Transfer in 5 Minutes with TensorFlow Hub & Magenta

Building a Neural Style Transfer app on iOS with PyTorch and CoreML

Fast Style Transfer Deeplearnjs

Neural Style Transfer

Reference:

Gatys, L.A., Ecker, A.S. and Bethge, M., 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576.

Babaeizadeh, M. and Ghiasi, G., 2018. Adjustable real-time style transfer. arXiv preprint arXiv:1811.08560.

Huang, H., Wang, H., Luo, W., Ma, L., Jiang, W., Zhu, X., Li, Z. and Liu, W., 2017. Real-time neural style transfer for videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 783–791).

Johnson, J., Alahi, A. and Fei-Fei, L., 2016, October. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision (pp. 694–711). Springer, Cham.

Li, Y., Wang, N., Liu, J. and Hou, X., 2017. Demystifying neural style transfer. arXiv preprint arXiv:1701.01036.