📂Materials AI

Moment Tensor Potential

Overview and Summary

The moment tensor potential is a type of 🔒(26/11/17)machine learning interatomic potential proposed by Alexander V. Shapeev in 2016. In the paper “Moment Tensor Potentials: a class of systematically improvable interatomic potentials”¹, he proposed describing the local atomic environment with moment tensors and constructing a basis of scalar polynomials satisfying 🔒(26/11/13)rotational invariance through tensor contraction.

Quantum mechanical calculations such as density functional theory are accurate, but they cost $\mathcal{O}(N^{3})$ with respect to the number of atoms $N$, so for large systems one uses 🔒(26/11/07)interatomic potentials. The author points out that 🔒(26/11/07)empirical potentials such as the 🔒(26/12/11)Lennard-Jones potential all have a fixed functional form and so their accuracy cannot be improved systematically, and proposes a potential that can approximate to any desired precision by increasing the basis. Here, the potential $V(\mathbf{u})$ for an 🔒(26/11/05)atomic environment $\mathbf{u} = (\mathbf{u}_{1}, \dots, \mathbf{u}_{n})$ must satisfy the following three conditions.

• (R1) The value is unchanged when the order of neighboring atoms is permuted (permutation invariance).
• (R2) For all $Q \in \operatorname{O}(3)$, $V(Q\mathbf{u}_{1}, \dots, Q\mathbf{u}_{n}) = V(\mathbf{u}_{1}, \dots, \mathbf{u}_{n})$ (rotation and reflection invariance).
• (R3) It is smooth when atoms cross the boundary of the cutoff radius.

The key idea is as follows. Since a smooth $V$ can be approximated by polynomials, one thinks of a way to construct polynomials that satisfy (R1) and (R2). The tool for this is the moment tensor.

$$ M_{\mu, \nu}(\mathbf{u}) = \sum_{i=1}^{n} \left| \mathbf{u}_{i} \right|^{2\mu} \mathbf{u}_{i}^{\otimes \nu} $$

Since a moment tensor is computed with a single sum, it takes linear time in the number of atoms $n$. By contracting the indices of these tensors with one another, one builds rotation-invariant scalars $B_{\alpha}$, and defines the potential as their linear combination. For a detailed explanation of $B_{\alpha}$, see the main text below.

$$ V(\mathbf{u}) = \sum_{\alpha \in \mathcal{A}} c_{\alpha} B_{\alpha}(\mathbf{u}) $$

The main results of the paper are as follows.

• Theorem 3.1: The $B_{\alpha}$ span the entire space of polynomials that are both permutation invariant and rotation invariant. That is, there exists a suitable index set $\mathcal{A}$ such that the interatomic potential $V$ can be expressed as a linear combination of the $B_{\alpha}$.

• Theorem 3.2: If the site energy of a simple tight-binding model is approximated by an invariant polynomial of degree $m$, the error decreases as $C\rho^{-m}$ $(\rho \gt 1)$.

• Since $V$ is linear in the coefficients $c_{\alpha}$, the coefficients are learned by linear regression.

• In actual implementations, to satisfy (R3), $\left| \mathbf{u}_{i} \right|^{2\mu}$ is replaced by a radial function $f_{\mu, \nu}(\left| \mathbf{u}_{i} \right|)$ multiplied by a 🔒(26/11/07)cutoff function.

Definition

Let the atomic positions of a system consisting of $N$ atoms be $\mathbf{x} = (\mathbf{x}_{1}, \dots, \mathbf{x}_{N}) \in (\mathbb{R}^{3})^{N}$, and let the 🔒(26/11/05)atomic neighborhood of atom $k$ with respect to the cutoff radius $r_{c}$ be $\mathcal{N}_{k}$.

$$ \mathcal{N}_{k} = \left\{ \mathbf{u}_{i} \right\}_{i \in \mathcal{I}_{k}}, \quad \mathbf{u}_{i} = \mathbf{x}_{i} - \mathbf{x}_{k}, \quad \mathcal{I}_{k} = \left\{ i : 0 \lt \left| \mathbf{x}_{i} - \mathbf{x}_{k} \right| \le r_{c} \right\} $$

An 🔒(26/11/07)interatomic potential $V$ expressed as follows, as a linear combination of the basis polynomials $B_{\alpha}$ defined below, is called a moment tensor potential (MTP).

$$ E(\mathbf{x}) = \sum_{k=1}^{N} V(\mathcal{N}_{k}), \qquad V(\mathcal{N}_{k}) = \sum_{\alpha \in \mathcal{A}} c_{\alpha} B_{\alpha}(\mathcal{N}_{k}) $$

Here $\mathcal{A}$ is a predetermined finite set of matrix indices $\alpha$, and $c_{\alpha} \in \mathbb{R}$ are coefficients determined by training.

Notation in the Paper

The paper writes the same content with the following notation.

Object	This article	Paper
Number of atoms	$N$	$N_{\text{tot}}$
Atomic position	$\mathbf{x}_{k}$	$x_{k}$
Cutoff radius	$r_{c}$	$R_{\text{cut}}$
Atomic neighborhood of atom $k$	$\mathcal{N}_{k}$ (set)	$Dx_{k}$ (tuple), atomic environment
Atomic neighborhood used as a variable	$\mathcal{N}_{k} = \left\{ \mathbf{u}_{i} \right\}_{i \in \mathcal{I}_{k}}$	$u = (u_{1}, \dots, u_{n})$
Norm of a vector	$\left\Vert \mathbf{u}_{i} \right\Vert$	$\left\vert u_{i} \right\vert$

The atomic environment and energy are as follows.

$$ Dx_{k} := (x_{i} - x_{k})_{1 \le i \le N_{\text{tot}},\ 0 \lt \left\vert x_{i} - x_{k} \right\vert \le R_{\text{cut}}} \tag{2.1} $$

$$ E(x) := \sum_{k=1}^{N_{\text{tot}}} V(Dx_{k}) \tag{2.2} $$

$$ V(u) = \sum_{\alpha \in A} c_{\alpha} B_{\alpha}(u) $$

Since the paper takes the atomic neighborhood as a tuple, it regards $V$ as a family of functions with different numbers of arguments, and instead requires the permutation invariance (R1) separately. If one takes it as a set as in this article, there is no ordering, so the permutation invariance is already built into the notation. Also, the paper develops the theory with a general spatial dimension $d$, but this article fixes $d = 3$.

Moment Tensor $M_{\mu, \nu}$

Fix one atom in a system consisting of $N$ atoms. Let the relative position vectors of the $n$ atoms within radius $r_{c}$ of the fixed atom be $\mathbf{u}_{i}$ $(i = 1, \dots, n)$, and let the tuple collecting them be $\mathbf{u} = (\mathbf{u}_{1}, \dots, \mathbf{u}_{n}) \in (\mathbb{R}^{3})^{n}$. For two integers $\mu$, $\nu \ge 0$, the $M_{\mu, \nu}$ defined as follows is called the moment polynomial of $\mathbf{u}$.

$$ M_{\mu, \nu}(\mathbf{u}) = \sum_{i = 1}^{n} \left| \mathbf{u}_{i} \right|^{2\mu} \mathbf{u}_{i}^{\otimes \nu} $$

Here $\mathbf{v} \otimes \mathbf{w}$ is the tensor product defined as below, and $\mathbf{v}^{\otimes \nu}$ is this repeated $\nu$ times.

$$ \mathbf{v} \otimes \mathbf{w} = \left[ v_{i}w_{j} \right] \in \mathbb{R}^{3 \times 3}, \quad i,j \in \set{1, 2, 3} $$

$$ \mathbf{v}^{\otimes \nu} = \left[ v_{i_{1}} \cdots v_{i_{\nu}} \right] \in \mathbb{R}^{\overbrace{3 \times \cdots \times 3}^{\nu}}, \quad i_{1}, \dots, i_{\nu} \in \set{1, 2, 3} $$

That is, $\mathbf{v}^{\otimes \nu}$ is a multidimensional array with $\nu$ indices. $\mu$ is a weight on the magnitude of each position vector, and $\nu$ determines the dimension of the moment tensor. As can be seen from the formula above, the dimension of the moment tensor depends only on $\nu$. When $\nu = 0$, it is defined to be $1$.

$\nu$	$\mathbf{v}^{\otimes \nu}$	Component	Object
$0$	$1$	$1$	scalar
$1$	$\mathbf{v}$	$v_{i}$	vector
$2$	$\mathbf{v} \otimes \mathbf{v} = \mathbf{v}\mathbf{v}^{\mathsf{T}}$	$v_{i} v_{j}$	$3 \times 3$ matrix
$3$	$\mathbf{v} \otimes \mathbf{v} \otimes \mathbf{v}$	$v_{i} v_{j} v_{k}$	$3 \times 3 \times 3$ array

Basis Polynomial $B_{\alpha}$

The inner product of two vectors is the value obtained by multiplying corresponding components and summing them all.

$$ \braket{\mathbf{u} \vert \mathbf{v}} = \mathbf{u} \cdot \mathbf{v} = u_{1}v_{1} + u_{2}v_{2} + u_{3}v_{3} = \sum_{i=1}^{3} u_{i} v_{i} \tag{asdf} $$

Similarly, we want to define a general operation that multiplies corresponding components of $k$ moment tensors $M$ and sums them all. Let $\alpha$ be a $k \times k$ symmetric matrix whose entries are nonnegative integers. Denote the diagonal entries by $\mu_{i} = \alpha_{ii}$, and the sum of all off-diagonal entries in row $i$ by $\nu_{i} = \sum_{j \ne i} \alpha_{ij}$. As one may guess, these are respectively the indices of the moment tensor $M_{\mu, \nu}$. The off-diagonal entry $\alpha_{ij}$ of $\alpha$ means the number of indices shared by the $i$th moment tensor and the $j$th moment tensor. Here, a shared index means the same subscript inside the summation sign, as on the right-hand side of equation $(asdf)$.

As a concrete example, consider $\alpha = \begin{bmatrix} \mu_{1} & 1 \\ 1 & \mu_{2} \end{bmatrix}$. Since the size of $\alpha$ is $2 \times 2$, this means there are two moment tensors, and since $\nu_{1} = \nu_{2} = 1$, each is a vector. For convenience, we omit $\mathbf{u}$.

$$ M_{\mu_{1}, 1}, M_{\mu_{2}, 1} \in \mathbb{R}^{3} $$

If $\alpha = \begin{bmatrix} \mu_{1} & 2 \\ 2 & \mu_{2} \end{bmatrix}$, this means there are two moment tensors which are matrices.

$$ M_{\mu_{1}, 2}, M_{\mu_{2}, 2} \in \mathbb{R}^{3 \times 3} $$

Finally, if $\alpha = \begin{bmatrix} \mu_{1} & 1 & 1 \\ 1 & \mu_{2} & 0 \\ 1 & 0 & \mu_{3} \end{bmatrix}$, this means there are three moment tensors, and since $\nu_{1} = 2$, $\nu_{2} = 1$, $\nu_{3} = 1$, they are one matrix and two vectors.

$$ M_{\mu_{1}, 2} \in \mathbb{R}^{3 \times 3}, \quad M_{\mu_{2}, 1}, M_{\mu_{3}, 1} \in \mathbb{R}^{3}, \quad $$

Index Set $\mathcal{B}$

Now, for a given $\alpha$, we define the index set $\mathcal{B} = \mathcal{B}(\alpha)$. $\mathcal{B}$ is the set of subscripts over which the summation sign is applied when multiplying corresponding components of the $k$ moment tensors and summing them all. In the explanation above, we said that the off-diagonal entry $\alpha_{ij}$ of $\alpha$ is the number of indices shared by the $i$th moment tensor and the $j$th moment tensor. If $\alpha = \begin{bmatrix} \mu_{1} & 2 \\ 2 & \mu_{2} \end{bmatrix}$, the two moment tensors share two indices, so let each of them be $\beta_{1}^{(1,2)}$, $\beta_{2}^{(1,2)}$. In general, the indices shared by the $i$th moment tensor and the $j$th moment tensor can be written as below. Since index sharing is symmetric, $\beta_{m}^{(j,i)} = \beta_{m}^{(i,j)}$.

$$ \beta_{1}^{(i,j)}, \dots, \beta_{\alpha_{ij}}^{(i,j)} $$

The components of the $i$th moment tensor are expressed as below. Since the notation is cumbersome, let us simply write the tuple of indices of the $i$th moment tensor as $b_{i}$.

$$ \left[ M_{\mu_{i}, \nu_{i}} \right]_{b_{i}} = \left[ M_{\mu_{i}, \nu_{i}} \right]_{\beta_{1}^{(i,1)}, \dots, \beta_{\alpha_{i1}}^{(i,1)}, \dots, \beta_{1}^{(i,i-1)}, \dots, \beta_{\alpha_{i,i-1}}^{(i,i-1)}, \beta_{1}^{(i,i+1)}, \dots, \beta_{\alpha_{i,i+1}}^{(i,i+1)}, \dots, \beta_{1}^{(i,k)}, \dots, \beta_{\alpha_{ik}}^{(i,k)} } $$

$$ b_{i} = (\beta_{1}^{(i,1)}, \dots, \beta_{\alpha_{i1}}^{(i,1)}, \dots, \beta_{1}^{(i,i-1)}, \dots, \beta_{\alpha_{i,i-1}}^{(i,i-1)}, \beta_{1}^{(i,i+1)}, \dots, \beta_{\alpha_{i,i+1}}^{(i,i+1)}, \dots, \beta_{1}^{(i,k)}, \dots, \beta_{\alpha_{ik}}^{(i,k)} ) $$

The indices of all the moment tensors can be written as $b = (b_{1}, b_{2}, \dots, b_{k})$. Since we assumed $3$ dimensions, each $\beta_{m}^{(i,j)}$ can take the values $1$, $2$, $3$, and let the set of all these possible combinations be $\mathcal{B}$.

$$ \mathcal{B} = \left\{ b = (b_{1}, \dots, b_{k}) : \beta_{m}^{(i,j)} \in \set{1, 2, 3},\ 1 \le i \lt j \le k,\ 1 \le m \le \alpha_{ij} \right\} $$

The definition is somewhat complicated in order to describe the index set formally, but it is not difficult once one looks at actual examples. If $\alpha = \begin{bmatrix} \mu_{1} & 1 \\ 1 & \mu_{2} \end{bmatrix}$, the moment tensors are two vectors and there is one shared index.

$$ b_{1} = (\beta_{1}^{(1,2)}) = b_{2} $$

$$ \mathcal{B} = \set{ \big((\beta_{1}^{(1,2)}), (\beta_{1}^{(1,2)})\big) : \beta_{1}^{(1,2)} \in \set{1, 2, 3} } = \set{ ((1),(1)), ((2),(2)), ((3),(3)) } $$

If $\alpha = \begin{bmatrix} \mu_{1} & 2 \\ 2 & \mu_{2} \end{bmatrix}$, the moment tensors are two matrices and there are two shared indices.

$$ b_{1} = (\beta_{1}^{(1,2)}, \beta_{2}^{(1,2)}) = b_{2} $$

$$ \begin{align*} \mathcal{B} &= \set{ \big( (\beta_{1}^{(1,2)}, \beta_{2}^{(1,2)}), (\beta_{1}^{(1,2)}, \beta_{2}^{(1,2)}) \big) : \beta_{1}^{(1,2)}, \beta_{2}^{(1,2)} \in \set{1, 2, 3} } \\ &= \big\{ \big( (1, 1), (1, 1) \big), \big( (1, 2), (1, 2) \big), \big( (1, 3), (1, 3) \big), \\ &\qquad \big( (2, 1), (2, 1) \big), \big( (2, 2), (2, 2) \big), \big( (2, 3), (2, 3) \big), \\ &\qquad \big( (3, 1), (3, 1) \big), \big( (3, 2), (3, 2) \big), \big( (3, 3), (3, 3) \big) \big\} \end{align*} $$

In any case, $\mathcal{B}$ is the set of all possible indices of all the moment tensors, and since it is a definition for the sake of definition, do not get too bogged down in the definition itself even if it is hard to grasp. Its meaning will become clear once you see below how the actual basis polynomials are summed.

Defining the Basis Polynomial via Tensor Contraction

Based on the definition of the index set above, the basis polynomial $B_{\alpha}$ is defined as follows.

$$ B_{\alpha} := \sum_{b \in \mathcal{B}} \prod_{i=1}^{k} \left[ M_{\mu_{i}, \nu_{i}} \right]_{b_{i}} $$

If $\alpha = \begin{bmatrix} \mu_{1} & 1 \\ 1 & \mu_{2} \end{bmatrix}$, the single index $\beta_{1}^{(1,2)}$ shared by the first moment tensor and the second moment tensor is all there is. The basis polynomial equals the inner product of two vectors.

$$ B_{\alpha} = \sum_{\beta_{1}^{(1,2)} = 1}^{3} \left[ M_{\mu_{1}, 1} \right]_{\beta_{1}^{(1,2)}} \left[ M_{\mu_{2}, 1} \right]_{\beta_{1}^{(1,2)}} = \braket{ M_{\mu_{1}, 1} \vert M_{\mu_{2}, 1} } = M_{\mu_{1}, 1} \cdot M_{\mu_{2}, 1} $$

If $\alpha = \begin{bmatrix} \mu_{1} & 2 \\ 2 & \mu_{2} \end{bmatrix}$, the two moment tensors share two indices, $\beta_{1}^{(1,2)}$ and $\beta_{2}^{(1,2)}$. The basis polynomial equals the (Frobenius) inner product of two matrices.

$$ B_{\alpha} = \sum_{\beta_{1}^{(1,2)} = 1}^{3} \sum_{\beta_{2}^{(1,2)} = 1}^{3} \left[ M_{\mu_{1}, 2} \right]_{\beta_{1}^{(1,2)}, \beta_{2}^{(1,2)}} \left[ M_{\mu_{2}, 2} \right]_{\beta_{1}^{(1,2)}, \beta_{2}^{(1,2)}} = \braket{ M_{\mu_{1}, 2} \vert M_{\mu_{2}, 2} }_{F} $$

If $\alpha = \begin{bmatrix} \mu_{1} & 1 & 1 \\ 1 & \mu_{2} & 0 \\ 1 & 0 & \mu_{3} \end{bmatrix}$, there are two indices, and it is the bilinear form of the two vectors $M_{\mu_{2}, 1}$, $M_{\mu_{3}, 1}$ and the matrix $M_{\mu_{1}, 2}$.

$$ \begin{align*} B_{\alpha} &= \sum_{\beta_{1}^{(1,2)} = 1}^{3} \sum_{\beta_{1}^{(1,3)} = 1}^{3} \left[ M_{\mu_{1}, 2} \right]_{\beta_{1}^{(1,2)}, \beta_{1}^{(1,3)}} \left[ M_{\mu_{2}, 1} \right]_{\beta_{1}^{(1,2)}} \left[ M_{\mu_{3}, 1} \right]_{\beta_{1}^{(1,3)}} \\ &= M_{\mu_{2},1}^{\mathsf{T}} M_{\mu_{1},2} M_{\mu_{3},1} \end{align*} $$

Representability Theorem

Statement

Let $\mathcal{P}$ be the set of all polynomials in the components of $n$ vectors $\mathbf{u}_{1}, \dots, \mathbf{u}_{n} \in \mathbb{R}^{3}$, that is, in $3n$ real variables. Define the subsets $\mathcal{P}_{\text{perm}}$, $\mathcal{P}_{\text{rot}}$ of $\mathcal{P}$ as follows.

$$ \begin{align*} \mathcal{P}_{\text{perm}} &= \set{ p \in \mathcal{P} : p(\mathbf{u}_{\sigma(1)}, \dots, \mathbf{u}_{\sigma(n)}) = p(\mathbf{u}_{1}, \dots, \mathbf{u}_{n}),\ \forall \sigma \in S_{n} } \\ \mathcal{P}_{\text{rot}} &= \set{ p \in \mathcal{P} : p(Q\mathbf{u}_{1}, \dots, Q\mathbf{u}_{n}) = p(\mathbf{u}_{1}, \dots, \mathbf{u}_{n}),\ \forall Q \in \operatorname{O}(3) } \end{align*} $$

That is, $\mathcal{P}_{\text{perm}}$ is the set of permutation invariant polynomials, and $\mathcal{P}_{\text{rot}}$ is the set of rotation and reflection invariant polynomials. Then the basis polynomials $B_{\alpha}$ form a spanning set of $\mathcal{P}_{\text{rot}} \cap \mathcal{P}_{\text{perm}}$. That is, any $p \in \mathcal{P}_{\text{rot}} \cap \mathcal{P}_{\text{perm}}$ can be expressed as a finite linear combination of the $B_{\alpha}$. However, that linear combination is in general not unique.

$$ p(\mathbf{u}_{1}, \dots, \mathbf{u}_{n}) = \sum_{\alpha \in \mathcal{A}} c_{\alpha} B_{\alpha}(\mathbf{u}_{1}, \dots, \mathbf{u}_{n}) $$

Explanation

This is Theorem 3.1 of the paper. It means that a polynomial that is permutation invariant as well as rotation and reflection invariant can be expressed exactly as a linear combination of the $B_{\alpha}$. The reason this theorem is the foundation of MTP is as follows. If the interatomic potential $V$ is assumed to be a smooth function, then $V$ can be approximated by polynomials by the Weierstrass approximation theorem. Here, the approximating polynomial can be chosen to have the same symmetry as $V$², and such a polynomial is expressed as a linear combination of the $B_{\alpha}$ by this theorem. Therefore, by taking $\mathcal{A}$ sufficiently large, $V$ can be approximated to any desired precision.

$$ V(\mathbf{u}) \approx \sum_{\alpha \in \mathcal{A}} c_{\alpha} B_{\alpha}(\mathbf{u}) $$

That each $B_{\alpha}$ belongs to $\mathcal{P}_{\text{rot}} \cap \mathcal{P}_{\text{perm}}$ follows immediately from the definition. The moment tensor $M_{\mu, \nu}$ is a sum over all neighboring atoms, so it is independent of the order of the neighboring atoms, and when $\mathbf{u}_{i}$ is replaced by $Q\mathbf{u}_{i}$, it rotates along as a tensor. Since $B_{\alpha}$ is a scalar obtained by contracting all the indices of such tensors without remainder, it is invariant for the same reason that the inner product of vectors is invariant under rotation.

Although they are called basis polynomials, the fact that the linear combination is not unique means that the $B_{\alpha}$ are not linearly independent, so they are not a basis in the sense of linear algebra. For example, permuting the rows and columns of $\alpha$ by the same permutation yields the same $B_{\alpha}$.