Differences Between the Monte Carlo Method and the Bootstrap
Overview
The Monte Carlo method repeats simulations with artificial data to verify new techniques, whereas the bootstrap tries to solve problems while cutting costs by resampling from actual data.
Definition
The monte Carlo method is a method that finds a point estimator for a target of interest through random number generation.
The bootstrap is a method that grasps the distribution of a target of interest through resampling from a sample.
Explanation
Both can be confusing in that they basically draw many times and run many trials, but only the process looks similar; their uses are entirely different.
First, the Monte Carlo method is used when we know some true value and want to see whether some statistic estimates it accurately. The tricky point in this explanation is, 'how on earth would we know the true value?' The answer is surprisingly simple: if the data was generated by the experimenter directly supplying the parameters and the distribution, then it is normal to know what the theoretical true value is. By repeating simulations with such artificial data, we can end up checking how close a new method or statistic is to reality.
The bootstrap, on the other hand, starts by drawing samples again from a given sample, thereby increasing the sample itself. In statistics, the more samples the better, so an increase in samples is itself a welcome thing. When running a single experiment is too expensive and difficult, or when no more data can be secured at all, such a method can be of great help to a project. The problem is how one can trust such a thing. Anyone who has properly studied statistics would naturally feel something is fishy just upon hearing it.
Naturally, to use the bootstrap, we need the assumption that the given sample at least reflects the characteristics of the population well. Even if it reflects them well, we must also take into account that biased data may occasionally arise. On top of that, the results obtained through the bootstrap only tell us the distribution; they cannot tell us exactly what the value we want to obtain is.
Suppose we use an estimator $\hat{ \alpha}$ to find the true value $\alpha$. The bootstrap will repeatedly obtain the estimator $\hat{ \alpha} ^{ \ast }$ of $\hat{ \alpha}$ through resampling. If we let the differences between the value we want to find and the estimators be, respectively, $$ \begin{cases} d = \hat{\alpha} - \alpha \\ d^{ \ast } = \hat{\alpha}^{ \ast } - \hat{ \alpha } \end{cases} $$ then their variances, with $\alpha$ and $\hat{ \alpha}$ treated as constants, become $$ \begin{cases} \Var (d) = \Var ( \hat{\alpha} - \alpha ) = \Var ( \hat{\alpha}) \\ \Var ( d^{ \ast } ) = \Var ( \hat{\alpha}^{ \ast } - \hat{ \alpha } ) = \Var ( \hat{\alpha}^{ \ast } ) \end{cases} $$ As a result, even if we find out $\Var (\hat{\alpha}^{ \ast })$, we cannot know what $\alpha$ is; we can only guess at its distribution.
For example, if we used the bootstrap on the regression coefficients, even though their standard errors can be obtained as fairly close values, we cannot know whether the coefficients themselves were properly obtained from a normal sample. If the obtained coefficients are not accurate, the t-test is also meaningless, and therefore we cannot know whether the regression coefficients are significant, so it is of no use whatsoever.
In conclusion, the Monte Carlo method and the bootstrap are not just a matter of having their own pros and cons; everything except the process is completely different.