A perspective of Intelligence System through Deep Embedded Clustering
By: Kristina P. Sinaga
Artificial intelligence (AI) growth massively, especially during these pandemic of COVID 19. People around the world tried their best to accelerate system by developing technologies. Virtual Reality, Augmentation reality, Artificial Intelligence, Image recognition etc. are technologies that relatively useful and will continuously be developed to achieve a perfect results in many applications that including intelligence system in the process. Machine learning as part of AI system have important roles to process data in order to produce meaningful results. While deep learning will be useful if data input is an image data. Recently, researchers develop an intelligence system by combining deep learning with unsupervised learning called deep embedding clustering. Deep embedding clustering (DEC) is proposed by Xie et al. [1], a method that simultaneously learns feature representations and cluster assignments using deep neural network (DNN). DEC learns a mapping from the data space to a lower-dimensional feature space in which it iteratively optimizes a clustering objective.
The idea of DEC is simply by transforming data into a new space with a nonlinear mapping. To achieved its goal, learning parameters and latent feature space are taking into account during the process of discovering a pattern. DEC has two phases, they are parameter initialization with a deep autoencoder and parameter optimization (i.e., clustering), where we iterate between computing an auxiliary target distribution and minimizing the Kullback-Leibler (KL) divergence to it. However, Deep embedded clustering (DEC) starts with pretraining an autoencoder and then removes the decoder. In this sense, we could say that these two component of autoencoder and clustering loss are essentials. The autoencoder essential because it is used to learn representations in unsupervised manner and the learned features can preserve intrinsic local structure in data. While clustering loss is responsible for manipulating embedded space in order to scatter embedded points.
References:
- Guo, Xifeng, et al. “Improved Deep Embedded Clustering with Local Structure Preservation.”Ijcai. 2017.
- Xu, Jie, et al. “Deep embedded multi-view clustering with collaborative training.”Information Sciences 573 (2021): 279-290.