In many Markov chain Monte Carlo problems, the target density function is known up to a normalization constant. In this paper, we take advantage of this knowledge to facilitate the convergence diagnostic of a Markov sampler by estimating the $L^1$ error of a kernel estimator. Firstly, we propose an estimator of the normalization constant which is shown to be asymptotically normal under mixing and moment conditions. Secondly, the $L^1$ error of the kernel estimator is estimated using the normalization constant estimator, and the ratio of the estimated $L^1$ error to the true $L^1$ error is shown to converge to 1 in probability under similar conditions. Thirdly, we propose a sequential plot of the estimated $L^1$ error as a tool to monitor the convergence of the Markov sampler. Finally, a 2-dimensional bimodal example is given to illustrate the proposal, and two Markov samplers are compared in the example using the proposed diagnostic plot.