Blog has been migrated!

The original blog has been migrated to WordPress!

新站開張!我原本的Blog已經搬到這邊啦!

Posted in 未分類 | Leave a comment

Logistics regression in Chinese

Here are recommended posts in Chinese for understanding logistics regressions. The readers can read them step by step. 

2. 你可能不知道的邏輯回歸(Logistics Regression), URL: https://taweihuang.hpd.io/2017/12/22/logreg101/
Posted in Data Mining, Machine Learning | Leave a comment

科技的進步,學無止境

在2006年左右,我還在唸嘉義大學數學系時,跟同學分工合作,用PHP 5+Dreamweaver 寫學校處室網站,那時候學校IT不給MySQL/Sql Server,我自己還默默用很簡單的檔案系統處理函數,定義好資料結構,一行一行把「最新消息」等訊息,存在單一檔案作為offline database使用(那時候學藝不精,不知道有Sqlite可以用)。就這樣把學校處室的消息發布功能做了出來,還具備CRUD功能。 

在那個大學年代,身兼學校的BBS系統站長,在那邊辛苦學習FreeBSD , Linux⋯當起MIS,三不五時就是去修伺服器,怎麼用fsck修硬碟, Make tools, build kernel 還自己去學怎麼剪RJ 45網路線,讓Server可以正常運作(可見那時候多窮,還要自己剪網路線)。
就這樣懵懵懂懂地大學畢業。
碩士班時期,實驗室還沒有積累一些關於data mining智慧資產,自己寫了一些演算法,像是PrefixSpan sequential pattern mining. 為了求效能,還全部用C++搭配STL寫。自己排crontab job,搭配bash script來產生自己要的實驗記錄。
博士班時期,老實說蠻精彩的。
當了三年的兼任助理,幫忙編hadoop教材…。為了自己的生計,還幫學校處室架設Wordpress,改別人的theme,甚至去看wordpress template怎麼寫,修掉別人模板的bug。也稍微懂得怎麼調整Apache的參數讓系統效能能夠跟得上來。
因為過去都用PHP寫網站,也順便接了幾個案子,像是做一些報名系統,多語系網站建置。
去當了幾家公司的兼任工程師,都是以C# + ASP.Net/MVC 作為主要工作技能。
曾經接了某個專利事務所的案子,幫別人寫Chrome外掛(後續做得不錯,顧客還有回鍋要我再幫忙開發,但是要顧學業,就婉拒了。)
曾幫博弈平台公司維護模組(這期間眼睛也去動手術了)。
為了打軟體競賽,還去用MongoDB,寫寫jQuery and Bootstrap。
為了做某公司的軟體專案,自己去玩Zebra ZPL印表機語言, WCF跟實作軟體序號與數位簽章演算法。
為了幫德國實驗室的同事做Big Data Benchmark,學怎麼用HBase & MapReduce API來做一些ETL工作(後來這也成為我的論文參考文獻之一)。
那時候政府還很流行Open Data,就去學架設CKAN,以及去研究HTTP協議,寫一些CKAN的prototype plugin。
出社會工作了,這些學到的技能也逐漸成為我的技術基石,不過是這些都是後話了。
Posted in 未分類 | 2 Comments

Raspberry Pi Camera + OpenCV

If you are interested in how to use Raspberry Pi Camera + OpenCV to capture images, here is a great post in Chinese. 

However, the post had something wrong. If you want to install OpenCV packages of Python 3, please use the command “apt-get install python3-opencv”. 
Posted in Python, Raspberry Pi | Leave a comment

FFMPEG: Decode and then encode frames to JPEG images

I’ve used FFMPEG library for a while. Actually, the FFMPEG library’s decoding process flow can be described as the following picture. If you want to read the videos and then save to jpeg file, you can take a look on my programming code (tested on FFMPEG ver. 4 library).

// VideoProcessing.cpp 
//
#pragma once
using namespace std;
#include <iostream>
extern "C"
{
#include <libavcodec/avcodec.h>
#include <libavdevice/avdevice.h>
#include <libavformat/avformat.h>
#include <libavfilter/avfilter.h>
#include <libavutil/avutil.h>
#include <libswscale/swscale.h>
#pragma comment(lib, "avcodec.lib")
#pragma comment(lib, "avdevice.lib")
#pragma comment(lib, "avformat.lib")
#pragma comment(lib, "avfilter.lib")
#pragma comment(lib, "avutil.lib")
#pragma comment(lib, "swscale.lib")
}

void SaveToJPEG(AVFrame* pFrame, const char * folderName, int index)

{

// Setup Output Path
char outFile[256] = { 0 };
sprintf_s(outFile, sizeof(outFile)/sizeof(outFile[0]), "%s\OutputImages-%d.jpg", folderName, index);


AVFormatContext* pFormatCtx = avformat_alloc_context();

// Setup the output format
pFormatCtx->oformat = av_guess_format("mjpeg", NULL, NULL);

// Initializext
if (avio_open(&pFormatCtx->pb, outFile, AVIO_FLAG_READ_WRITE) < 0) {

printf("Couldn't open output file.");

return;

}

// Get a new Stream from the indicated format context
AVStream* pAVStream = avformat_new_stream(pFormatCtx, 0);

if (pAVStream == NULL) {

return;

}

// Find encoder from the codec identifier.

AVCodec* pCodec = avcodec_find_encoder(pFormatCtx->oformat->video_codec);

if (!pCodec) {
printf("Codec not found.");
return;
}
// Setup the codec context
AVCodecContext* codecCtx = avcodec_alloc_context3(pCodec);
codecCtx->codec_id = pFormatCtx->oformat->video_codec;
codecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
codecCtx->pix_fmt = AV_PIX_FMT_YUVJ420P;
codecCtx->width = pFrame->width;
codecCtx->height = pFrame->height;
codecCtx->time_base = AVRational{ 1,25 };

// Open the codec
if (avcodec_open2(codecCtx, pCodec, NULL) < 0) {

printf("Could not open codec.");

return;

}
// assign the codec context to the stream parameters.
avcodec_parameters_from_context(pAVStream->codecpar, codecCtx);


//Write Header
avformat_write_header(pFormatCtx, NULL);

int y_size = (codecCtx->width) * (codecCtx->height);

// assign large enough space
AVPacket pkt;

av_new_packet(&pkt, y_size);

int got_picture = 0;

// Use avcodec_send_frame()/avcodec_receive_packet() instead
int ret = avcodec_send_frame(codecCtx, pFrame);

if (ret < 0) {

printf("Encode Error.n");

return;

}
else
{
ret = avcodec_receive_packet(codecCtx, &pkt);

ret = av_write_frame(pFormatCtx, &pkt);

}

av_packet_unref(&pkt);

//Write Trailer
av_write_trailer(pFormatCtx);

printf("Encode Successful.n");

avcodec_close(codecCtx);

avio_close(pFormatCtx->pb);

avformat_free_context(pFormatCtx);
avcodec_free_context(&codecCtx);
}

int main(int argc, char * argv[])
{

if (argc < 2) {
cout << "You need to specify a media file." << endl;
cout << "Command line : VideoProcessing.exe [input_video_path] [output_folder]" << endl;
return -1;
}

AVFormatContext* pFormatContext = avformat_alloc_context();
if (!pFormatContext) {
cout << "ERROR could not allocate memory for Format Context" << endl;
return -1;
}

if (avformat_open_input(&pFormatContext, argv[1], NULL, NULL) != 0) {
cout << "ERROR could not open the file" << endl;
return -1;
}

if (avformat_find_stream_info(pFormatContext, NULL) < 0) {
cout << "ERROR could not get the stream info" << endl;
return -1;
}

// Initialize the codec, paramters for subsequent useage.
AVCodec* pCodec = NULL;
AVCodecParameters* pCodecParameters = NULL;
int videoStreamIndex = -1;

for (int i = 0; i < pFormatContext->nb_streams; i++)
{
AVCodecParameters* pLocalCodecParameters = NULL;
// Read the codec parameters corresponding to each stream.
pLocalCodecParameters = pFormatContext->streams[i]->codecpar;

AVCodec* pLocalCodec = NULL;
pLocalCodec = avcodec_find_decoder(pLocalCodecParameters->codec_id);

if (pLocalCodec == NULL)
{
cout << "[ERROR] Cannot find the codec" << endl;
}

if (pLocalCodecParameters->codec_type == AVMEDIA_TYPE_VIDEO)
{
if (videoStreamIndex == -1)
{
videoStreamIndex = i;
pCodec = pLocalCodec;
pCodecParameters = pLocalCodecParameters;
}
}
}

AVCodecContext* pCodecContext = avcodec_alloc_context3(pCodec);
if (pCodecContext == NULL)
{
cout << "Fail to allocate the memoery to the Codec Context." << endl;
return -1;
}

if (avcodec_parameters_to_context(pCodecContext, pCodecParameters) < 0)
{
cout << "failed to copy codec params to codec context" << endl;
return -1;
}

if (avcodec_open2(pCodecContext, pCodec, NULL) < 0)
{
cout << "failed to open codec through avcodec_open2" << endl;
return -1;
}

AVFrame* pFrame = av_frame_alloc();
if (!pFrame)
{
cout << "failed to allocated memory for AVFrame" << endl;
return -1;
}

AVPacket* pPacket = av_packet_alloc();
if (!pPacket)
{
cout << "failed to allocated memory for AVPacket" << endl;
return -1;
}

int indexOfFrame = 0;
while (av_read_frame(pFormatContext, pPacket) >= 0)
{
// if it's the video stream
if (pPacket->stream_index == videoStreamIndex) {
int response = avcodec_send_packet(pCodecContext, pPacket);
if (response < 0)
{
break;
}
else
{
response = avcodec_receive_frame(pCodecContext, pFrame);

if (response >= 0)
{
indexOfFrame++;
SaveToJPEG(pFrame, argv[2], indexOfFrame);
}
}
}
av_packet_unref(pPacket);
// Limit the number of output frame to be 5.
if (indexOfFrame == 5)
{
break;
}

}
// https://ffmpeg.org/doxygen/trunk/group__lavc__packet.html#ga63d5a489b419bd5d45cfd09091cbcbc2
avformat_close_input(&pFormatContext);
av_frame_free(&pFrame);
avcodec_free_context(&pCodecContext);
}

Reference:

  1. FFMPEG libav decode note
  2. FFMPEG libav tutorial
  3. 用AVCodecParameters代替AVCodecContext
  4. 用FFmpeg保存JPEG图片

Posted in C, FFMPEG | Leave a comment

The introduction for JavaScript ES6’s classes

Do not understand the class features of JavaScript ES6?

You can take a look on this introduction in Chinese. 
Posted in JavaScript | Leave a comment

Speed up ActiveMQ performance by 25x

Sometimes we encounter the performance issue for persistent messaging using ActiveMQ. 
Here’s the best practice on Linux system to accelerate it. 

https://blog.christianposta.com/activemq/speeding-up-activemq-persistent-messaging-performance-by-25x/

Posted in ActiveMQ | Leave a comment

使用 git worktree 建立多個工作區

Reference: 

Posted in git | Leave a comment

Materials for Neural Network

Some materials are enclosed here.
1. 深度學習(二): 反向傳播 URL:http://chansh518.github.io/deep%20learning/2016/08/08/Deep-Learning-Notes-Backpropagation.html
2. 一文看懂常用的梯度下降算法 URL: https://blog.csdn.net/u013709270/article/details/78667531
3. 邏輯回歸代價函數及其梯度下降公式 URL: https://blog.csdn.net/Mr_HHH/article/details/78934793
4. The Back Propagation Algorithm. URL: https://page.mi.fu-berlin.de/rojas/neural/chapter/K7.pdf
5. Derivation of Back Propagation Algorithm for Forward Neural Networks. URL:  http://www.cs.put.poznan.pl/pliskowski/pub/teaching/eio/lab1/eio-supplementary.pdf
6. 凸優化 梯度下降。URL: http://www.hanlongfei.com/凸优化/2015/09/29/cmu-10725-gradient/
7. Geadient Descent demystified. URL:  https://towardsdatascience.com/gradient-descent-demystified-bc30b26e432a
8. An introduction to gradient descent and linear regression. URL: https://spin.atomicobject.com/2014/06/24/gradient-descent-linear-regression/

Posted in 未分類 | Leave a comment

It’s time to push my career forward

After a long time, almost 3 months, I’ve done my PhD degree. In the past, I could not image that I get my PhD. Thanks for my supervisor’s and family’s great supports. I did it!
Through the training of PhD, making things clear and be systematic are not the difficult parts anymore for me. But, I deeply felt that doing research tasks and engineering works was the most difficult part!

Now, I am a software engineer of an international corporate.
For me, doing both engineering and research work is better than doing only researches!
Now, it’s time to push my career forward.

Posted in 未分類 | Leave a comment