《On clustering using random walks》阅读笔记

1. 问题建模

1.1 问题描述

let $G(V,E,\omega )$ be a weighted graph, $V$ is the set of nodes, $E$ is the edge between nodes in $V$, $\omega$ is the function $\omega ：E\to {\mathbb{R}}^n$, that measures the simularity between pairs of items(a higher value means more similar).

$$p_{ij}=\frac{\omega (i,j)}{d_i}$$
$$d_i=\sum _{k=1}^n\omega (i,k)$$

$M^G\in {\mathbb{R}}^{n\times n}$ is the associated transition matrix,
$$M_{ij}^G=\{\begin{array}{ll}{p}_{ij}& ⟨i,j⟩\in E\\ 0& \text{otherwise}\end{array}$$

Question:

1. $\omega$表示节点之间的相似性，实际上我们只有无向图，表示节点之间是否有连接，怎么通过已有的信息构建$\omega$
   answer: 这里的相似度可以认为是节点之间边的权值，所以$M_{ij}^G$可以认为是认为是以邻接矩阵操作后的数据。

这里的内容比较坑，我在论文中一直找不到关于$P_{\text{visit}}^k(i)$是怎么计算的，在这里卡了好久好久。

在原文中的描述是这样的：

Now, denote by $P_{visit}^k(i)\in {\mathbb{R}}^n$ the vector whose j-th component is the probability that a random walk originating at i will visit node j in its k-th step. Thus,$P_{visit}^k(i)$ is the i-th row in the matrix $(M^G{)}^k$, the k’th power of $M^G$.

现在我们知道$M^G$是怎样计算的，但是$(M^G{)}^k$呢，在原文中的描述是’'the k’th power of $M^G$", 我理解的应该是原有矩阵$M^G$的k次方（矩阵的乘法）。

$P_{visit}^k(i)$ is the i-th row in the matrix $(M^G{)}^k$,

$$P_{visit}^k(i)=(M^G{)}_i^k$$
( M G ) k = { P v i s i t k ( 1 ) T , P v i s i t k ( 2 ) T , … , P v i s i t k ( n ) T } (M^G)^k=\{P^k_{visit}(1)^{\mathbf{T}}, P^k_{visit}(2)^{\mathbf{T}}, \dots, P^k_{visit}(n)^{\mathbf{T}}\} (MG)k={Pvisitk​(1)T,Pvisitk​(2)T,…,Pvisitk​(n)T}

Notice: 其实到这里，和马尔可夫聚类算法（MCL）是一样的。MCL是不断迭代，知道矩阵不再改变，这里作者考虑到计算复杂，采用前k次计算结果的和来作为替代。

We now offer two methods for performing the edge separation, both based on deterministic analysis of random walks.

边缘分离，锐化

NS: Separation by neighborhood similarity.

CE: Separation by circular escape.

the weighted neighborhood ： 加权领域
bipartite subgraph

$$P_{\text{visit}}^{\le k}(v)=\sum _{i=1}^k{P}_{\text{visit}}^i(v)$$

2. NS: Separation by neighborhood similarity.

Now, in order to estimate the closeness of the two node $v$ and $u$ , we fix some small k(eg. k = 3) and compare $P_{\text{visit}}^{\le k}(v)$ and $P_{\text{visit}}^{\le k}(u)$. The smaller the difference, the greater the intimacy between $u$ and $v$.

$$NS(G)\stackrel{dfn}{=}{G}_s(V,E,{\omega }_s)$$,
where $$\forall ⟨v,u⟩\in E,{\omega }_s(u,v)=si{m}^k(P_{visit}^{\le k}(v),P_{visit}^{\le k}(u))$$

$si{m}^k(x,y)$ is some similarity measure of the vectors $\mathrm{x}$ and $\mathrm{y}$, whose value increases as $\mathrm{x}$ and $\mathrm{y}$ are more similar.

$si{m}^k(x,y)$ the suitable choose:
$$f^k(x,y)\stackrel{dfn}{=}\exp (2k-\Vert x-y{\Vert }_{L_1})-1\text{\hspace{1em}(1)}$$
$$\Vert x-y{\Vert }_{L_1}=\sum _{i=1}^n|x_i-y_i|$$

another choose is:
$$\cos (x,y)=\frac{(x,y)}{\sqrt{(x,x)}.\sqrt{(y,y)}}\text{\hspace{1em}(2)}$$
where (·,·) denotes inner-product.(内积)

3.2 CE: Separation by circular escape.

3.3 代码实现

import numpy as np


def markovCluster(adjacencyMat, dimension, numIter, power=2, inflation=2):
    columnSum = np.sum(adjacencyMat, axis=0)
    probabilityMat = adjacencyMat / columnSum

    # Expand by taking the e^th power of the matrix.
    def _expand(probabilityMat, power):
        expandMat = probabilityMat
        for i in range(power - 1):
            expandMat = np.dot(expandMat, probabilityMat)
        return expandMat

    expandMat = _expand(probabilityMat, power)

    # Inflate by taking inflation of the resulting
    # matrix with parameter inflation.
    def _inflate(expandMat, inflation):
        powerMat = expandMat
        for i in range(inflation - 1):
            powerMat = powerMat * expandMat
        inflateColumnSum = np.sum(powerMat, axis=0)
        inflateMat = powerMat / inflateColumnSum
        return inflateMat

    inflateMat = _inflate(expandMat, inflation)

    for i in range(numIter):
        expand = _expand(inflateMat, power)
        inflateMat = _inflate(expand, inflation)
    print(inflateMat)
    print(np.zeros((7, 7)) != inflateMat)


if __name__ == "__main__":
    dimension = 4
    numIter = 10
    adjacencyMat = np.array([[1, 1, 1, 1],
                             [1, 1, 0, 1],
                             [1, 0, 1, 0],
                             [1, 1, 0, 1]])

    # adjacencyMat = np.array([[1, 1, 1, 1, 0, 0, 0],
    #                          [1, 1, 1, 1, 1, 0, 0],
    #                          [1, 1, 1, 1, 0, 0, 0],
    #                          [1, 1, 1, 1, 0, 0, 0],
    #                          [0, 1, 0, 0, 1, 1, 1],
    #                          [0, 0, 0, 0, 1, 1, 1],
    #                          [0, 0, 0, 0, 1, 1, 1],
    #                          ])
    markovCluster(adjacencyMat, dimension, numIter)

[[1.00000000e+000 1.00000000e+000 1.00000000e+000 1.00000000e+000]
 [5.23869755e-218 5.23869755e-218 5.23869755e-218 5.23869755e-218]
 [0.00000000e+000 0.00000000e+000 0.00000000e+000 0.00000000e+000]
 [5.23869755e-218 5.23869755e-218 5.23869755e-218 5.23869755e-218]]
[[ True  True  True  True]
 [ True  True  True  True]
 [False False False False]
 [ True  True  True  True]]

可以从中得到聚类效果 { { 1 ， 2 ， 4 } ， { 3 } } \{\{1，2，4\}，\{3\}\} {{1，2，4}，{3}}

谱聚类
MCL
MCL GitHub