COMET

prior research

• MF Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30-37.
  • 내적(Inner Product)으로 목표 사용자와 목표 아이템의 차원별 2차 상호작용(element-wise interaction)을 모델링할 뿐, 차원 간 교차 상호작용을 모델링하지 못함
• ConvNCF He, X., Du, X., Wang, X., Tian, F., Tang, J., & Chua, T. S. (2018). Outer product-based neural collaborative filtering. arXiv preprint arXiv:1808.03912.
  • 외적(outer product)으로 목표 사용자와 목표 아이템의 2차 교차 상호작용(cross-dimensional interaction)을 생성하나 이들을 둘러싼 맥락의 관계 구조를 모델링하지 못함
• FM Rendle, S. (2010, December). Factorization machines. In 2010 IEEE International conference on data mining (pp. 995-1000). IEEE.
  • 목표 사용자와 목표 아이템 뿐만 아니라 그 맥락 정보까지 활용하여 고차 상호작용(high-order interaction)을 모델링하나 맥락 정보(context)로서 목표 사용자와 목표 아이템의 부가 정보(side information)을 동원함

idea

• COMET(COnvolutional diMEnsion inTeraction): 목표 사용자와 목표 아이템의 과거 상호작용 이력을 맥락 정보(context)로 활용하여 그 고차 교차 상호작용(high-order cross-dimensional interaction)을 모델링하는 잠재요인 모형

notation

• $u=1,2,\cdots,M$: user idx
• $i=1,2,\cdots,N$: item idx
• $\mathbf{X}^{\mathrm{(user)}} \in \mathbb{R}^{M \times M}$: user one-hot matrix
• $\mathbf{X}^{\mathrm{(item)}} \in \mathbb{R}^{N \times N}$: item one-hot matrix
• $\mathbf{Y} \in \mathbb{R}^{M \times N}$: user-item interaction matrix
• $\mathbf{p}_{u} \in \mathbb{R}^{K}$: user id embedding vector
• $\mathbf{q}_{i} \in \mathbb{R}^{K}$: item id embedding vector
• \(\mathbf{E}_{u} \in \mathbb{R}^{\vert \mathcal{R}_{u}^{+} \setminus \{i\} \vert \times K}\): history embedding map of user $u$
• \(\mathbf{E}_{i} \in \mathbb{R}^{\vert \mathcal{R}_{i}^{+} \setminus \{u\} \vert \times K}\): history embedding map of item $i$
• $\mathbf{g}_{u}$: history interaction vector of user $u$
• $\mathbf{g}_{i}$: history interaction vector of item $i$
• $\mathbf{u}_{u} \in \mathbb{R}^{K}$: user history embedding vector
• $\mathbf{v}_{i} \in \mathbb{R}^{K}$: item history embedding vector
• $\hat{y}_{u,i}$: interaction probability of user $u$ and item $i$

function

• index embedding:

  \[\begin{aligned} \mathbf{p}_{u} &=\mathbf{W}\cdot\mathbf{X}_{u*}^{\mathrm{(user)}}\\ \mathbf{q}_{i} &=\mathbf{W}\cdot\mathbf{X}_{i*}^{\mathrm{(item)}} \end{aligned}\]

• history interaction modeling:

  \[\begin{aligned} \mathbf{u}_{u} &=\cdots \left(\left\{\mathbf{q}_{j} \mid \forall j \in \mathcal{R}_{u}^{+} \setminus \{i\}\right\}\right)\\ \mathbf{v}_{i} &= \cdots \left(\left\{\mathbf{p}_{v} \mid \forall v \in \mathcal{R}_{i}^{+} \setminus \{u\}\right\}\right) \end{aligned}\]

• matrix factorization:

  \[\begin{aligned} \mathbf{z}_{u,i} &= \left(\mathbf{p}_{u} + \mathbf{u}_{u}\right) \odot \left(\mathbf{q}_{i} + \mathbf{v}_{i}\right) \end{aligned}\]

• predict interaction probability of user $u$ and item $i$:

  \[\begin{aligned} \hat{y}_{u,i} &= \sigma\left(\mathbf{W} \cdot \mathbf{z}_{u,i}\right) \end{aligned}\]

history interaction modeling

• history embedding maps:

  \[\begin{aligned} \mathbf{E}_{u} = \begin{bmatrix} \mathbf{q}_{1 \in \mathcal{R}_{u}^{+} \setminus \{i\}}\\ \mathbf{q}_{2 \in \mathcal{R}_{u}^{+} \setminus \{i\}}\\ \vdots\\ \mathbf{q}_{j \in \mathcal{R}_{u}^{+} \setminus \{i\}} \end{bmatrix},\quad \mathbf{E}_{i} = \begin{bmatrix} \mathbf{p}_{1 \in \mathcal{R}_{i}^{+} \setminus \{u\}}\\ \mathbf{p}_{2 \in \mathcal{R}_{i}^{+} \setminus \{u\}}\\ \vdots\\ \mathbf{p}_{v \in \mathcal{R}_{i}^{+} \setminus \{u\}} \end{bmatrix} \end{aligned}\]

• CNN:

  \[\begin{aligned} \mathbf{g}_{u} &= \mathrm{flatten}\left[\mathrm{conv}_{\mathrm{ReLU}}(\mathbf{E}_{u})\right]\\ \mathbf{g}_{i} &= \mathrm{flatten}\left[\mathrm{conv}_{\mathrm{ReLU}}(\mathbf{E}_{i})\right] \end{aligned}\]
  • \(\mathcal{W} \in \mathbb{R}^{\vert \mathcal{R}^{+} \setminus \{u,i\} \vert \times H}\): Kernel Window Dimension
  • \(H \in \{1,8,32,128\}\): kernel window size
  • the number of filters per kernel size is $8$
  • max pooling applied

• generate history embedding:

  \[\begin{aligned} \mathbf{u}_{u} &=\mathrm{mlp}_{\mathrm{ReLU}}(\mathbf{g}_{u})\\ \mathbf{v}_{i} &=\mathrm{mlp}_{\mathrm{ReLU}}(\mathbf{g}_{i}) \end{aligned}\]