As an interdisciplinary talent of data science and biomedicine based in Taipei, Taiwan.
Enjoy learning new technologies and solving problems, as well as being passionate in making the world better.
Data Pipeline (ETL)
1. Build efficient web crawlers to extract public data to databases mainly using Python Requests, Beautiful Soup, Selenium.
2. Clean and transform data into useful information by Python Pandas, NumPy.
3. Create API to connect with databases by Python Flask.
4. Analyze information via commercial tools, including Splunk, ELK, Tableau, Qlik Sense and also visualize in websites.
Web Development
1. Style static websites by HTML, CSS.
2. Advanced functions including animations, dynamic tables and responsive web design (RWD) by JavaScript, jQuery and Bootstrap.
3. Query data through PHP for dynamic visualization charts, including Google Charts, Apache ECharts, Chart.js.
Databases
1. Data storage and query with relational databases, including MySQL, MariaDB and also No-SQL MongoDB.
2. Hadoop Ecosystem including Spark, Scala, Hive, Flume, Sqoop.
Linux
1. Linux including CentOS, Debian and TurnKey LAMP Stack.
2. Setup and maintain services, applications, websites.
3. Monitor status and analyse service logs.
4. Shell Scripting for automation of common tasks.
Real Time Hacks
rth is a cross-platform, real-time, customizable solution, that can easily trace out hacker attacks and non-standard connections without looking through millions of log data.
As product manager and data engineer intern of Foresii corp. I worked with other data engineers to built a prototype from scratch.
Trend Analysis - Job Bank
Analyze top 4 job banks in Taiwan, including 104, 1111, 518, yes123.
Focused on jobs related to big data and software engineers, we built crawlers to extract new data every week and automatically transform to useful information, which will update websites' charts and API data.
As a project coordinator, I worked closely with the manager to built up the data pipeline and integrate several different codes to complete the automation.
Sentiment Analysis - COVID-19
Public data about COVID-19 from 2 major news websites and 2 major forum in Taiwan were gathered by web crawlers that we created, and we used python jieba's Chinese text segmentation module to get separated words for sentiment analysis.
As a data engineer intern of BiMAP corp. I worked with another intern to create web crawlers and sentiment analysis model.
PN109 Student Attendance Record Analysis
Analysed with Tableau
Knowing each student's behavior by Analyzing and visualizing the student's punch in/out record along with IP address and
class information.
GoNOW
A Traveler's all-in-one mobile application, features including Travel Schedule system, Celebrity Recommendation system,
and Travel Companion system. Not only can it recommend the best option through AI, but it also integrates with
third-party booking apps.
Prototype designed with Affinity Designer / Illustrator
(Co-designed by Xavier, James and Peter.)
