You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following is the flow as compared to SfSNet we are adding.
23
+

24
+
25
+
Illumination model based on Normal and Spherical Harmonics is not accurate and cannot model the illumination estimation.
26
+
Hence, we are proposing new illumination model, which works on top of Spherical Harmonics and Normal based
22
27
Measure issue with reconstructin faces using Shading, Albedo, Normal and Spherical Harmonics is little variations in normal and spherical harmonics causing different illumination.
23
28
24
29
Estimating Normal and Spherical Harmonics with slight error causes different illumination and expected.
25
-
In order to overcome this problem, we would like to using correcting networks
30
+
In order to overcome this problem, we propose new shading model (let's say residue) which works along with traditional illumination model and captures the illumination details missed by traditional model.
31
+
32
+
We propose different methods to capture this residue using following methods.
33
+
We also propose new illumination models, first to correct shading using latent lighting and second to generating shading directly without use of Spherical Harmonics.
26
34
27
35
### Shading Correcting Network
28
36
1_Shading_Correcting works on priciple built on top of tradition shading generation using normal and spherical harmonics
29
37
We generate Shading and then rectify it using latent ligthing generated from Normal, Albedo and Image features.

43
+
#### Interpolation on synthetic data
44
+

45
+
#### More training samples
46
+

47
+
31
48
### Latent Shading Generation
32
49
2_Latent_Shading_Gen works on principle to avoid generation of Spherical Harmonics and instead generate shading directly using Normal, Albedo and Image features. And then use this latent lighting along with Normal to generate shading

81
+
#### Interpolation on synthetic data
82
+

83
+
#### More training samples
84
+

85
+
86
+
### 5_Shading_Residual_Albedo_GAN
87
+
Problem with shading residue method is domain gap of Synthetic and Real albedo being generated. Hence, we use GAN to generate robust and albedo from synthetic domain which has real ground truth.

93
+
#### Interpolation on synthetic data
94
+

95
+
#### More training samples
96
+

97
+
#### More training samples with No Albedo loss
98
+
due to lack of albedo loss, intensity of albedo is pushed into residue
99
+

100
+
101
+
### 6_Shading_Resdiual_GAN_Separate_Train
102
+
In method 5, we train GAN based albedo generation and residue network along with each other. In this method, We first train GAN based albedo generation and then we fix GAN based method and then only train residue network.
103
+
104
+
### 7_Shading_Residual_GAN_train_2
105
+
Method 7 is slight variation of method 6 with different Discriminator
106
+
107
+
## Results samples comparison across the experiments
0 commit comments