(Mark) Haoyue Dai    

I'm currently a senior student majoring in CS at IEEE Honor Class, Shanghai Jiao Tong University (SJTU), graduating in June 2021. I'm also a member of Zhiyuan Honor Program of Engineering. I was an exchange student at the University of Washington.

My research interests are centered around AI interpretability, causality, and robust machine learning. My passion to explainable AI started from a group led by Prof. Quanshi Zhang at John Hopcroft Center. In that group I actively explored new methods of interpretation analysis to transform black-box networks to semantically explainable models. To further explain mathematically, since summer 2020 I've been working on causal discovery guided by Principal Researcher Justin Ding at Microsoft Research Asia (MSRA).

Update: I'll be starting my Master's at Philosophy, CMU this coming Fall.

Email  /  CV  /  GitHub

profile photo
Selected Programs
What do CNN neurons learn: Visualization & Clustering
Haoyue Dai
supervised by Prof. John Hopcroft, Fall 2019
arxiv / slides / code

In this study, we address the problem of interpreting a CNN from the aspects of the input image's focus and preference, and the neurons' domination, activation and contribution to a concrete final prediction. Specifically, we use two techniques - visualization and clustering to tackle the problems above. Visualization means the method of gradient descent on image pixel, and in clustering section two algorithms are proposed to cluster respectively over image categories and network neurons. Experiments and quantitative analyses have demonstrated the effectiveness of the two methods in explaining the question: what do neurons learn.

Interpretation of Speech Recognition Algorithms
Die Zhang, Haoyue Dai, Xinzhe Cao, Da Huo
supervised by Prof. Quanshi Zhang, Spring 2019
project page / code

By analyzing the mainstream speech recognition algorithms, we aim to fabricate a model to quantitatively characterize the "importance" of different parts of a voice, and further simplify speech recognition networks. I proposed a novel masking approach: not the noise linear superposition, but each points' value dispersed along neighboring frequency domain. I designed the entropy-based regularizer, worked out the differentiable approximation and optimized it in C++ for parallel acceleration. I also reconstructed the letter-based gated ConvNets wav2letter frame, and developed a series of well-packaged utilities like IFFT speech regeneration, noise coverage, mask separation, parameter visualization, etc.

Multi-category Classification: Basic Methods & Implementation from Scratch
Haoyue Dai
course project of Machine Learning, Spring 2020, offered by Prof. Quanshi Zhang
report1 / report2 / code1 / code2

This is the project for CS385 Machine Learning course. In project1, I implemented and optimized four different techniques (logistic regression, support vector machine, linear discriminant analysis and neural networks) from scratch, with effectiveness almost reaching mainstream machine learning packages. Experiment results and observations testifying algorithm principles based on MNIST handwritten digits dataset are shown in report. In project2, experiments are conducted on two CNN architectures and four different scaled datasets. I tried to compare the performances between different models, different datasets and different optimizer scheme. I then analyzed the feature maps using PCA, visualization methods, grad-CAM, etc.

Poem Inspire: An Image-Poem Coupled Search & Generation Engine
Haoyue Dai, Zhongye Wang, Jingyu Li, Haoping Chen
course project of EE208, Fall 2018, offered by Prof. Ya Zhang and Prof. Dazhi He
report / demo unavailable now :( / code

This is an integrated poem engine with features of searching, exhibition, imagination, recommendation, and image-poem generation, etc. I provided lexical semantic prediction and expansion model to give synonym clustering between classical and modern Chinese, conducted a Recurrent Neural Network model independently, which can generate classical Chinese quatrain from images, with expansion from keywords, and fine-tuned a deep coupled visual-poetic embedding model by multi-adversarial training, which can generate Chinese modern poems from image. Images' features will be extracted from convolutional networks first, and be used as poetic clues to generate poems from two discriminative networks.

Realtime Traffic Cone Detection
Haoyue Dai
applied on SJTU Racing Team's autonomous car, Spring 2019
dataset / code

This project focuses on real time traffic cone detection and has been applied on SJTU Racing Team's autonomous car, which performs well on Fomula Student China (FSC) 2019. I annotated a traffic cone dataset first, and fine-tuned a YOLOv3 network with mAP reaching 98%, recall 98%, precision 99%, and speed 60 fps on 1080Ti. Acceleration over constraints on TX2 and collaboration with SLAM has been designed for practical racing.


thanks jon!
my best friends 毛毛&豆豆