Index tracking is a passive investment strategy in which an investor purchases a set of assets to mimic a market index. The tracking error, the difference between the performances of the index and the portfolio, may be minimized by buying all the assets contained in the index. However, this strategy results in a considerable amount of transaction cost and, accordingly, decreases the return of the constructed portfolio. On the other hand, a portfolio with small cardinality may result in a poor out-of-sample performance. Of interest is, thus, the minimization of tracking error, while keeping the number of assets invested in small (i.e., sparse). In this paper, we develop a tracking portfolio model that addresses the conflicting requirements above by a combination of L0- and L2-norms. While L2-norm regularizes the overdetermined system to impose smoothness (and hence better out-of-sample) performance and shrinks the solution to the equally-weighted dense portfolio, L0-norm imposes a cardinality constraint that achieves sparsity (and hence lower transaction cost). We propose a heuristic method for parameter estimation in this model, which combines greedy search with an analytical formula embedded in it. We demonstrate that a sparse portfolio can achieve good tracking and generalization performance on historic data of weekly and daily returns on the Nikkei 225 index and its constituent companies.