Sheng-Syun Shen

Hi! This is Sheng-Syun Shen.

Currently drafted into the substitute military service in Hsinchu, Taiwan.

About Me

2014/09 - 2016/06 Graduate Institute of Communication Engineering, National Taiwan University, Taiwan.
M.S. in Electrical Engineering
Speech Processing and Machine Learning Laboratory
Advisor: Lin-Shan Lee

2010/09 - 2014/06 Department of Electrical Engineering, National Taiwan University, Taiwan.
B.S. in Electrical Engineering

My research interests include Natural Language Processing, Speech Processing, Machine Learning, Information Retrieval and Question Answering.

For more information about me, please refer to the CV.

Research

Machine Comprehension on TOEFL Test

Proposed multi-hop neural attention models for TOFEL listening comprehension test.
Designed a sentence-level attention mechanism to reduce the impact of recognition error on language understanding.
Publication: ISCA Interspeech 2016 as full paper (one of the 12 finalists for the best student paper award) [pdf]

Neural Attention Models for Sequence Classification

Proposed an innovative recurrent neural network architecture with an attention mechanism for sequence classification and achieved better performance in key term extraction and dialogue act classification than previous work.
Collected a new data set (290,000 articles) from StackOverflow for keyterm extraction.
Publication: ISCA Interspeech 2016 as full paper [pdf]

Online Learning - Lecture Linking and Prerequisite Prediction

Designed a system for organizing online courses by linking lectures with similar contents and reordering the lectures according to their prerequisite relationship.
Proposed a word embedding approach that extracts feature vectors from documents while preserving the hypernym information by incorporating Wordnet database.
Publication: ISCA Interspeech 2015 as full paper [pdf]

Online Learning - Slide and Lecture Alignment

Built a system to align audio and slides to help people learn by video more efficiently.
Exploited global structure information using structured support vector machine (SSVM) to achieve a 5.25% improvement in alignment accuracies.
Publication: ISCA Interspeech 2014 as full paper [pdf]

Projects

Vehicle Detection using Faster Region-based Convolutional Neural Networks (2016)

In this project, we proposed two approaches for vehicle detection: You Only Look Once (YOLO) and Faster R-CNN. Both methods are end-to-end detector and achieve real-time vehicle detection. We trained the two models with PASCAL VOC 2007 dataset and did an experiment on imLab testing dataset, achieving accuracy of 0.07098 for YOLO and 0.63576 for Faster R-CNN in terms of F1 score.

Teamwork with Yen-Chen Wu and Ching-Ning Chen.

Result: [report]

Emoticon Prediction using Recurrent Neural Networks (2016)

The project is about predicting authors' emotion according to the sentences they wrote, and each sentence contains texts and an emoticon. Emoticons can be viewed as authors' emotion when they wrote the sentence. We handled this problem by utilizing deep learning techniques. We encoded sentences into a sequence of vectors and formulate the project as a multi-class classification problem.

Teamwork with Yu-Hsuan Wang and Chia-Hsing Hsu.

Result: [report]

Information Retrieval and Text Classification on Mandarin Chinese News Report (2016)

In this project, we proposed some approaches for information retrieval and text classification, which are two sub-tasks in Natural Language Processing (NLP).

Teamwork with Wei-Tse Lee and Ching-Ta Wu.

Result: [report]

Visual Question Answering (2016)

In this project, we solved multiple choices question according to their corresponding images. To correctly answer questions, the proposed system needs to solve a number of sub-problems in Natural Language Processing and Computer Vision, in addition to being able to perform some kind of "common-sense" reasoning. It needs to localize the subject being referenced, and needs to detect objects.

Demo: [website] [slide]

Handwritten Chinese Character Recognition based on Convolutional Neural Networks (2015)

We used support vector machine (SVM) and convolutional neural networks (CNN) to solve this Chinese character recognition task. To cope with the lack of training data, we tried to increase the data by adding virtual examples with rotation and masking. Experiments showed that the Caffe model architecture performed the best on most of the features we extracted.

Teamwork with Hung-Tsung Lu and Chih-Hsiang Yang.

Result: [report]

Friend Recommendation using Matrix Factorization and Singular Value Decomposition (2015)

The goal of this project is to find out the subsequent adopters in a social communication graph according to the given idea adoption records (user logs) and idea initiators. We explored many models, such as Hottest Recommendation, Matrix Factorization, Singular Value Decomposition, Neural Network and Local Community Detection. We also evaluated our methods based on F1-Score.

Teamwork with Hsiang-Hung Lu and Yi-Hsiu Liao.

Result: [report]

Publications

Conference Papers

Bo-Hsiang Tseng, Sheng-Syun Shen, Hung-Yi Lee, and Lin-Shan Lee, "Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine," in Proc. Interspeech, San Francisco, USA, September 2016. [pdf] [poster]
Sheng-Syun Shen and Hung-Yi Lee, "Neural Attention Models for Sequence Classification: Analysis and Application to Key Term Extraction and Dialogue Act Detection," in Proc. Interspeech, San Francisco, USA, September 2016. [pdf] [poster]
Sheng-Syun Shen, Hung-Yi Lee, Shang-Wen Li, Victor Zue, and Lin-Shan Lee, "Structuring Lectures in Massive Open Online Courses (MOOCs) for Efficient Learning by Linking Similar Sections and Predicting Prerequisite," in Proc. Interspeech, Dresden, Germany, September 2015. [pdf] [poster]
Han Lu, Sheng-Syun Shen, Sz-Rung Shiang, Hung-Yi Lee, and Lin-Shan Lee, "Alignment of Spoken Utterances with Slide Content for Easier Learning with Recorded Lectures using Structured Support Vector Machine (SVM)," in Proc. Interspeech, Singapore, September 2014. [pdf] [poster]

Thesis

Sheng-Syun Shen, "More Efficient Learning by Structuring, Classifying and Understanding Lectures in Online Courses," in Department of Communication Engineering, National Taiwan University, Taiwan, June 2016. [pdf] [defense]