Bayesian hierarchical models are powerful tools for learning common latent features across multiple data sources. The Hierarchical Dirichlet Process (HDP) is invoked when the number of latent components is a priori unknown. While there is a rich literature on finite sample properties and performance of hierarchical processes, the analysis of their frequentist posterior asymptotic properties is still at an early stage. Here we establish theoretical guarantees for recovering the true data generating process when the data are modeled as mixtures over the HDP or a generalization of the HDP, which we term boosted because of the faster growth in the number of discovered latent features. By extending Schwartz's theory to partially exchangeable sequences we show that posterior contraction rates are crucially affected by the relationship between the sample sizes corresponding to the different groups. The effect varies according to the smoothness level of the true data distributions. In the supersmooth case, when the generating densities are Gaussian mixtures, we recover the parametric rate up to a logarithmic factor, provided that the sample sizes are related in a polynomial fashion. Under ordinary smoothness assumptions more caution is needed as a polynomial deviation in the sample sizes could drastically deteriorate the convergence to the truth.

Posterior asymptotics for boosted Hierarchical Dirichlet Process mixtures / Catalano, Marta; De Blasi, Pierpaolo; Lijoi, Antonio; Pruenster, Igor. - In: JOURNAL OF MACHINE LEARNING RESEARCH. - ISSN 1532-4435. - 23:80(2022), pp. 1-23.

Posterior asymptotics for boosted Hierarchical Dirichlet Process mixtures

Catalano, Marta;
2022

Abstract

Bayesian hierarchical models are powerful tools for learning common latent features across multiple data sources. The Hierarchical Dirichlet Process (HDP) is invoked when the number of latent components is a priori unknown. While there is a rich literature on finite sample properties and performance of hierarchical processes, the analysis of their frequentist posterior asymptotic properties is still at an early stage. Here we establish theoretical guarantees for recovering the true data generating process when the data are modeled as mixtures over the HDP or a generalization of the HDP, which we term boosted because of the faster growth in the number of discovered latent features. By extending Schwartz's theory to partially exchangeable sequences we show that posterior contraction rates are crucially affected by the relationship between the sample sizes corresponding to the different groups. The effect varies according to the smoothness level of the true data distributions. In the supersmooth case, when the generating densities are Gaussian mixtures, we recover the parametric rate up to a logarithmic factor, provided that the sample sizes are related in a polynomial fashion. Under ordinary smoothness assumptions more caution is needed as a polynomial deviation in the sample sizes could drastically deteriorate the convergence to the truth.
2022
Bayesian asymptotics, Dirichlet process, hierarchical process, nonparametric density estimation, partial exchangeability, Pitman-Yor process, posterior contraction rates
Posterior asymptotics for boosted Hierarchical Dirichlet Process mixtures / Catalano, Marta; De Blasi, Pierpaolo; Lijoi, Antonio; Pruenster, Igor. - In: JOURNAL OF MACHINE LEARNING RESEARCH. - ISSN 1532-4435. - 23:80(2022), pp. 1-23.
File in questo prodotto:
File Dimensione Formato  
jmlr_20-1474_published.pdf

Open Access

Tipologia: Versione dell'editore
Licenza: Creative commons
Dimensione 368.89 kB
Formato Adobe PDF
368.89 kB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11385/232219
Citazioni
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 0
  • OpenAlex ND
social impact