mirror of https://github.com/ZHKKKe/MODNet.git
update local webcam video matting demo and README
parent
c51ece7232
commit
a8e7a8504d
|
|
@ -10,7 +10,7 @@
|
|||
</p>
|
||||
|
||||
<div align="center">This is the official project of our paper <b>Is a Green Screen Really Necessary for Real-Time Portrait Matting?</b></div>
|
||||
<div align="center">MODNet is a <b>trimap-free</b> model for portrait matting in <b>real time</b> (on a single GPU).</div>
|
||||
<div align="center">MODNet is a <b>trimap-free</b> model for portrait matting in <b>real time</b> under <b>changing scenes</b>.</div>
|
||||
|
||||
|
||||
---
|
||||
|
|
@ -23,7 +23,7 @@
|
|||
|
||||
## Video Matting Demo
|
||||
We provide two real-time portrait video matting demos based on WebCam.
|
||||
If you have an Ubuntu system, we recommend you to try the [offline demo](demo/video_matting) to get a higher *fps*. Otherwise, you can access the [online Colab demo](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing).
|
||||
If you have an Ubuntu system, we recommend you to try the [offline demo](demo/video_matting/webcam) to get a higher *fps*. Otherwise, you can access the [online Colab demo](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing).
|
||||
|
||||
|
||||
## Image Matting Demo
|
||||
|
|
|
|||
|
|
@ -1,18 +1,17 @@
|
|||
## MODNet - WebCam-Based Portrait Video Matting Demo
|
||||
This is a MODNet portrait video matting demo based on WebCam. It will call your local WebCam and display the matting results in real time.
|
||||
This is a MODNet portrait video matting demo based on WebCam. It will call your local WebCam and display the matting results in real time. The demo can run under CPU or GPU.
|
||||
|
||||
### 1. Requirements
|
||||
The basic requirements for this demo are:
|
||||
- Ubuntu System
|
||||
- WebCam
|
||||
- Nvidia GPU with CUDA
|
||||
- Python 3+
|
||||
|
||||
**NOTE**: If your device does not satisfy the above conditions, please try our [online Colab demo](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing).
|
||||
|
||||
|
||||
### 2. Introduction
|
||||
We use ~400 unlabeled video clips (divided into ~50,000 frames) downloaded from the internet to perform SOC to adapt MODNet to the video domain. Nonetheless, due to insufficient labeled training data (~3k labeled foregrounds), our model may still make errors in portrait semantics estimation under challenging scenes. Besides, this demo does not currently support the OFD trick, which will be provided soon.
|
||||
We use ~400 unlabeled video clips (divided into ~50,000 frames) downloaded from the internet to perform SOC to adapt MODNet to the video domain. **Nonetheless, due to insufficient labeled training data (~3k labeled foregrounds), our model may still make errors in portrait semantics estimation under challenging scenes.** Besides, this demo does not currently support the OFD trick, which will be provided soon.
|
||||
|
||||
For a better experience, please:
|
||||
|
||||
|
|
@ -33,18 +32,18 @@ We recommend creating a new conda virtual environment to run this demo, as follo
|
|||
2. Download the pre-trained model from this [link](https://drive.google.com/file/d/1Nf1ZxeJZJL8Qx9KadcYYyEmmlKhTADxX/view?usp=sharing) and put it into the folder `MODNet/pretrained/`.
|
||||
|
||||
|
||||
3. Create a conda virtual environment named `modnet-webcam` and activate it:
|
||||
3. Create a conda virtual environment named `modnet` (if it doesn't exist) and activate it:
|
||||
```
|
||||
conda create -n modnet-webcam python=3.6
|
||||
source activate modnet-webcam
|
||||
conda create -n modnet python=3.6
|
||||
source activate modnet
|
||||
```
|
||||
|
||||
4. Install the required python dependencies (here we use PyTorch==1.0.0):
|
||||
```
|
||||
pip install -r demo/video_matting/requirements.txt
|
||||
pip install -r demo/video_matting/webcam/requirements.txt
|
||||
```
|
||||
|
||||
5. Execute the main code:
|
||||
```
|
||||
python -m demo.video_matting.webcam
|
||||
python -m demo.video_matting.webcam.run
|
||||
```
|
||||
|
|
@ -19,8 +19,17 @@ torch_transforms = transforms.Compose(
|
|||
print('Load pre-trained MODNet...')
|
||||
pretrained_ckpt = './pretrained/modnet_webcam_portrait_matting.ckpt'
|
||||
modnet = MODNet(backbone_pretrained=False)
|
||||
modnet = nn.DataParallel(modnet).cuda()
|
||||
modnet.load_state_dict(torch.load(pretrained_ckpt))
|
||||
modnet = nn.DataParallel(modnet)
|
||||
|
||||
GPU = True if torch.cuda.device_count() > 0 else False
|
||||
if GPU:
|
||||
print('Use GPU...')
|
||||
modnet = modnet.cuda()
|
||||
modnet.load_state_dict(torch.load(pretrained_ckpt))
|
||||
else:
|
||||
print('Use CPU...')
|
||||
modnet.load_state_dict(torch.load(pretrained_ckpt, map_location=torch.device('cpu')))
|
||||
|
||||
modnet.eval()
|
||||
|
||||
print('Init WebCam...')
|
||||
Loading…
Reference in New Issue