Max-pooling is a standard layer in neural network for dimension reduction. However, taking the max value in each kernel may not keep the feature of previous layers. We study how we can retain important features by replacing the max-pooling layers with singular value decomposition (SVD). We further explore how the SVD based dimension reduction affects the performance of medical image analysis tasks.
Second Order Optimization in Deep Learning
The first-order stochastic gradient descent is usually used to train deep neural networks. Recent progress in the second-order optimization methods such as Kronecker-factored Approximate Curvature (K-FAC) suggests a way to improve the convergence rate with reasonable overheads. We use such second-order methods to enhance the efficiency of deep learning based medical image analysis on parallel computers.
Strongly-Weakly Labels in X-ray Classification
We aim to test the hypothesis that re-casting multi-label classification as an objection detection problem will significantly improve performance with medical imaging and improve data efficiency.
Parallel Self Updating Process for Clustering
The Self Updating Process (SUP) is a powerful statistical-based clustering analysis tool. SUP can provide robust clustering results along with an auto-identified number of clusters. We refactored the update scheme of SUP to fit many-core GPU architecture. The parallel SUP has been applied to 3D lung CT image segmentation and particle classification in cryo-electron microscopy images. The multiple GPU results also show that SUP can handle large datasets, and its scalability is nearly linear.
Model robustness and generalization ability are both important topics in machine learning. It remains unclear why a deep learning model is successful in one domain dataset that can fail in another domain dataset. We address this issue by considering approaches such as the generative adversarial network (GAN), regularity, co-training, and incremental learning.
Large Scale Deep Learning
Training deep learning models requires high computational power, a single model training process with a few GPUs can take hours or even days. Our target is to provide a solution accelerating the model training iteration by using computer clusters with many GPUs. We focus on optimizing data storage, training data pipeline, and inter-GPU data transfer.