I have read the paper (2018)Entire Space Mulit-Task Model: An Effective Approach for Estimationg Post-Click Conversion Rate, and found there is an inconsistency in the model. So, I point out it in this blog, hoping I would have chance to discuss it with the authors later. The core idea of the model can be summarized as the following formula:

\[\underbrace{p(y=1,z=1 \vert x)}_{pCTCVR} = \underbrace{p(y=1 \vert x)}_{pCTR} \times \underbrace{p(z=1 \vert y=1, x)}_{pCVR} \qquad (1)\]

So, the author designs two tasks to learn pCTR and pCVR simultaneously, in which the two models share the embeddings so that pCVR task can make use of the information from pCTR task. The motivation is to solve the sample selection bias and data sparsity problems at the same time.

The author try to persuade us the formula (1) can explain the mechanism that the main task is to learn pCVR，and the auxiliary task is learn pCTR. However, the code of the ESMM shows that the main task also learns from samples in the impression-space(entire-space), but not the click-space. So, it should use $p(z=1 \vert x)$, instead of $p(z=1 \vert y=1, x)$, to explain the model. Then, the formula (1) can be fixed as following:

\[\underbrace{p(y=1,z=1 \vert x)}_{pCTCVR} \approx \underbrace{p(y=1 \vert x)}_{pCTR} \times \underbrace{p(z=1 \vert x)}_{pCVR} \qquad (2)\]

Formula (2) is not that perfect, as there is an approximation. I try to explain the inconsistency from the perspective of the authors. They found that an multi-tasks model can improve the estimation of the pCVR in the case of Taobao Recommendation. But, they also want to find an theory to prove the approach can work generally, which is formula (1).

However, what I want to diss is that they don’t point it out and try to cover up this. I also read some discussion from the community that ESMM doesn’t work in their cases. If the inconsistency did not exist, the paper would be very good as the practice mentioned is very remarkable and impressive for recommendation. The inconsistency makes me puzzled, so I discuss it with my colleagues and finally to make it clear.

Details are very important and pay attention to details to accomplish great things.