Improved MDL Estimators Using Local Exponential Family Bundles Applied to Mixture Families

The MDL estimators for density estimation, which are defined by two-part codes for universal coding, are analyzed. We give a two-part code for mixture families whose regret is close to the minimax regret, where regret of a code with respect to a target family ℳ is the difference between the codelength of the code and the ideal codelength achieved by an element in ℳ. Our code is constructed using a probability density in an enlarged family of ℳ (a bundle of local exponential families of ℳ) for data description. This result gives a tight upper bound on the risk of the MDL estimator defined by the two-part code, based on the theory introduced by Barron and Cover in 1991.