Skip to content

Commit d93d4df

Browse files
authored
Update README.md
Adding detailed result
1 parent 5cd3f40 commit d93d4df

1 file changed

Lines changed: 82 additions & 1 deletion

File tree

README.md

Lines changed: 82 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,95 @@ residue of illumination missed by SfSNet. Figure 6 shows new illumination model
1919

2020
![Proposed Illumination Model](report/images/method/new_illumination.png)
2121

22+
Following is the flow as compared to SfSNet we are adding.
23+
![SfSNet model](report/images/arch/our_arch.png)
24+
25+
Illumination model based on Normal and Spherical Harmonics is not accurate and cannot model the illumination estimation.
26+
Hence, we are proposing new illumination model, which works on top of Spherical Harmonics and Normal based
2227
Measure issue with reconstructin faces using Shading, Albedo, Normal and Spherical Harmonics is little variations in normal and spherical harmonics causing different illumination.
2328

2429
Estimating Normal and Spherical Harmonics with slight error causes different illumination and expected.
25-
In order to overcome this problem, we would like to using correcting networks
30+
In order to overcome this problem, we propose new shading model (let's say residue) which works along with traditional illumination model and captures the illumination details missed by traditional model.
31+
32+
We propose different methods to capture this residue using following methods.
33+
We also propose new illumination models, first to correct shading using latent lighting and second to generating shading directly without use of Spherical Harmonics.
2634

2735
### Shading Correcting Network
2836
1_Shading_Correcting works on priciple built on top of tradition shading generation using normal and spherical harmonics
2937
We generate Shading and then rectify it using latent ligthing generated from Normal, Albedo and Image features.
3038

39+
#### Architecture
40+
![Architecture](report/images/out/1_shading_correcting/arch.png)
41+
#### Interpolcation on real data
42+
![Interpolation on real data](report/images/out/1_shading_correcting/res_1_shading_correcting_real.png)
43+
#### Interpolation on synthetic data
44+
![Interpolation on synthetic data](report/images/out/1_shading_correcting/res_1_shading_correcting_syn.png)
45+
#### More training samples
46+
![More training samples](report/images/out/1_shading_correcting/train_out.png)
47+
3148
### Latent Shading Generation
3249
2_Latent_Shading_Gen works on principle to avoid generation of Spherical Harmonics and instead generate shading directly using Normal, Albedo and Image features. And then use this latent lighting along with Normal to generate shading
50+
51+
#### Architecture
52+
![Architecture](report/images/out/2_latent_shading_gen/arch.png)
53+
#### Interpolation on real data
54+
![Interpolation on real data](report/images/out/2_latent_shading_gen/res_2_latent_shading_real.png)
55+
#### Interpolation on synthetic data
56+
![Interpolation on synthetic data](report/images/out/2_latent_shading_gen/res_2_latent_shading_gen_syn.png)
57+
#### More training samples
58+
![More training samples](report/images/out/2_latent_shading_gen/train_out.png)
59+
60+
### 3_Shading_Residue
61+
This method predicts shading residue and adds into traditional shading model.
62+
New shading residue model is capturing details missed by traditional shading model.
63+
64+
#### Architecture
65+
![Architecture](report/images/out/3_shading_res/arch.png)
66+
#### Interpolation on real data
67+
![Interpolation on real data](report/images/out/3_shading_res/res_3_shading_res_real.png)
68+
#### Interpolation on synthetic data
69+
![Interpolation on synthetic data](report/images/out/3_shading_res/res_3_shading_residual_syn.png)
70+
#### More training samples
71+
![More training samples](report/images/out/3_shading_res/train_out.png)
72+
73+
### 4_Shading_Albedo_Residue
74+
This method is next step of Shading residue method which add manual control by adding predicted
75+
residue into Shading and subtracting from Albedo. We can refer this method to be baesd on
76+
Robinhood principle- which takes from albedo and gives to shading.
77+
78+
![Architecture](report/images/out/4_shading_albedo_res/arch.png)
79+
#### Interpolation on real data
80+
![Interpolation on real data](report/images/out/4_shading_albedo_res/res_4_shading_albedo_res_real.png)
81+
#### Interpolation on synthetic data
82+
![Interpolation on synthetic data](report/images/out/4_shading_albedo_res/res_4_shading_albedo_res_syn.png)
83+
#### More training samples
84+
![More training samples](report/images/out/4_shading_albedo_res/train_out.png)
85+
86+
### 5_Shading_Residual_Albedo_GAN
87+
Problem with shading residue method is domain gap of Synthetic and Real albedo being generated. Hence, we use GAN to generate robust and albedo from synthetic domain which has real ground truth.
88+
89+
#### Architecture
90+
![Architecture](report/images/out/5_gan_albedo/arch.png)
91+
#### Interpolation on real data
92+
![Interpolation on real data](report/images/out/5_gan_albedo/res_gan_real.png)
93+
#### Interpolation on synthetic data
94+
![Interpolation on synthetic data](report/images/out/5_gan_albedo/res_gan_real.png)
95+
#### More training samples
96+
![More training samples](report/images/out/5_gan_albedo/train_out.png)
97+
#### More training samples with No Albedo loss
98+
due to lack of albedo loss, intensity of albedo is pushed into residue
99+
![More training samples with No albedo Loss](report/images/out/5_gan_albedo/train_out_no_albedo_loss.png)
100+
101+
### 6_Shading_Resdiual_GAN_Separate_Train
102+
In method 5, we train GAN based albedo generation and residue network along with each other. In this method, We first train GAN based albedo generation and then we fix GAN based method and then only train residue network.
103+
104+
### 7_Shading_Residual_GAN_train_2
105+
Method 7 is slight variation of method 6 with different Discriminator
106+
107+
## Results samples comparison across the experiments
108+
![Real Sample 1](report/images/out/compare_all/real_1.png)
109+
![Real Sample 2](report/images/out/compare_all/real_2.png)
110+
![Real Sample 3](report/images/out/compare_all/real_3.png)
111+
![Real Sample 4](report/images/out/compare_all/real_4.png)
112+
![Synthetic Sample 1](report/images/out/compare_all/syn_1.png)
113+

0 commit comments

Comments
 (0)