2016年8月5日 星期五

Install CKAN 2.5 on Ubuntu 14.04 LTS

Author: Jyun-Yao Huang (Allen; allen501pc@gmail.com)
System Environment: Ubuntu Linux 14.04 LTS (64bits)

1. Update the metadata of packages

$ sudo apt-get update

2. Install apache2, nginx, apache’s module wsgi and the library of postgres- libpq5.

$ sudo apt-get install -y nginx apache2 libapache2-mod-wsgi libpq5

3. Download deb-package of python-ckan.
There are two ways to get CKAN deb package.
First, here you can download my provided deb package in the cloud storage. URL: https://drive.google.com/file/d/0B6PKt2-xNai_QlNfdEZfY05JRlE/view?usp=sharing
Or, download it from CKAN official site.

$ wget http://packaging.ckan.org/python-ckan_2.5-trusty_amd64.deb

4. Install the package.

$ sudo dpkg -i python-ckan_2.5-trusty_amd64.deb

5. Install postgresql.

$ sudo apt-get install postgresql

6. Check the encoding format of Postgresql is UTF-8.

$ sudo -u postgres psql -l

7. Create the database account and the corresponding database. In this case, the account is ckan_default and the database is also called ckan_default.
Please remember your password of your database account.

$ sudo -u postgres createuser -S -D -R -P ckan_default
$ sudo -u postgres createdb -O ckan_default ckan_default -E utf-8

8. Edit the file at /etc/ckan/default/production.ini with super user’s permission. Check the settings below:

sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default

Please replace words in bold with the password of your created account for Postgresql.

9. Create the fold for CKAN storage. Type the following command to create storage.

$ sudo mkdir /var/lib/ckan
$ chown –R www-data:www-data /var/lib/ckan

10. Edit the file at /etc/ckan/default/production.ini with super user’s permission. Check the settings below:

ckan.storage_path = /var/lib/ckan

11. Download Solr 5.4.1
Warning: Form the CKAN official guide, if you install solr-jetty in Ubuntu 14.04, the JSP of Solr will not be supported. We have to install Solr.
There are two ways to get Solr package.
First, here you can download my provided Solr package in the cloud storage. URL: https://drive.google.com/open?id=0B6PKt2-xNai_Tm9sSnJoZmx1VW8
Or download it from Solr's site.

$ wget http://apache.mirror1.spango.com/lucene/solr/5.4.1/solr-5.4.1.tg

12. Install Solr 5.4.1

$ tar xzf solr-5.4.1.tgz solr-5.4.1/bin/install_solr_service.sh --strip-components=2
$ sudo chmod +x install_solr_service.sh
$ sudo ./install_solr_service.sh solr-5.4.1.tgz -f

13. Prepare the CKAN’s schema for Solr
In this case, we prepare a core named ckan for Solr.

$ sudo su
root$ mkdir /var/solr/data/ckan
root$ touch /var/solr/data/ckan/core.properties
root$ cp -a /opt/solr/server/solr/configsets/basic_configs/conf /var/solr/data/ckan
root$ mv /var/solr/data/ckan/conf/schema.xml /var/solr/data/ckan1/conf/schema.xml.orig
root$ ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml /var/solr/data/ckan/conf/schema.xml
root$ chown -R solr:solr /var/solr
root$ exit

14. [Support of Chinese Segmentation] Download the libraries for supporting Chinese segmentation in Solr. There are two packages: mmseg4j-core-x.y.z.jar and mmseg4j-lang-x.y.z.jar to be downloaded. I’ve prepared the zipped file – mmseg4j.zip at https://drive.google.com/open?id=0B6PKt2-xNai_a1luR20wQ0JtenM.

15. [Support of Chinese Segmentation] After downloading mmseg4j.zip and unzipping it into mmseg4j-core-1.10.0.jar and mmseg4j-solr-2.3.0.jar, upload JARs under the folder - /opt/solr/server/solr-webapp/webapp/WEB-INF/lib.

16. [Support of Chinese Segmentation]Modify the schema file at /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml to let Solr support Chinese segmentation.

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="com.chenlb.mmseg4j.solr.MMSegTokenizerFactory" mode="max-word"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ASCIIFoldingFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="com.chenlb.mmseg4j.solr.MMSegTokenizerFactory" mode="max-word"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.SnowballPorterFilterFactory" language="English" protected="protwords.txt"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ASCIIFoldingFilterFactory"/>
</analyzer>
</fieldType>

17. Restart Solr.

$ sudo service solr restart

18. Serve the following URL to create the core called ckan via your browser.
http://127.0.0.1:8983/solr/admin/cores?action=CREATE&name=ckan&configSet=ckan

19. Edit the file at /etc/ckan/default/production.ini with super user’s permission. Check the setting for Solr below:

solr_url = http://127.0.0.1:8983/solr/ckan

In this case, ckan is the core we used in Solr, so we add ckan as suffix of the URL.

20. Initialize your database. We have to use VirtualEnv which is a virtual environment written in Python. Here, (virtualEnv) is the mark for denoting you are in VirtualEnv.

$ . /usr/lib/ckan/default/bin/activate
$ (virtualEnv) cd /usr/lib/ckan/default/src/ckan
$ (virtualEnv) paster db init -c /etc/ckan/default/production.ini
$ (virtualEnv) deactivate

21. Create Administrator for CKAN.

$ . /usr/lib/ckan/default/bin/activate
$ (virtualEnv) cd /usr/lib/ckan/default/src/ckan
$ (virtualEnv) paster sysadmin add admin -c /etc/ckan/default/production.ini
$ (virtualEnv) deactivate

22. Restart apache2 and nginx.

$ sudo service apache2 restart
$ sudo service nginx restart

After that, you will get the following CKAN site.

Image 39

Support of SSL

With the support of SSL, we can enhance the security of CKAN.

First, we have to apply for an SSL certificate. For freelancers, we can request the SSL certificate from Let’s Encrypt. The method of applying such certificate has been introduced in the official site of Let’s Encrypt. Interested readers can refers to its documents.

After getting an SSL certificate, please following the instructions in the post: Setting up CKAN with SSL.

REFERENCE

  1. CKAN, “Installing CKAN from Package,” CKAN. [Online]. Available: http://docs.ckan.org/en/latest/maintaining/installing/install-from-package.html
  2. CKAN, “Installing CKAN from Source,” CKAN. [Online]. Available: http://docs.ckan.org/en/latest/maintaining/installing/install-from-source.html
  3. Koen Vlasinkel, “How to Install Solr 5.2.1 on Ubuntu 14.04,” Digital Ocean Community. [Online]. Available: https://www.digitalocean.com/community/tutorials/how-to-install-solr-5-2-1-on-ubuntu-14-04
  4. Hvwaldow, “How to Upgrade to SOLR 5.3.1 on Debian Jessie for Use with CKAN,” CKAN. [Online]. Available: https://gist.github.com/hvwaldow/67fecf80a9790b5c9153#filesolr-upgrade-md
  5. CKAN2 安裝教學. URL: http://ckan-docs-tw.readthedocs.io/zh_TW/2.4/install.html
  6. CKAN, “Setting up CKAN with SSL,”. [Online]. Available: https://github.com/ckan/ckan/wiki/SSL

2016年7月21日 星期四

[Wordpress] A Method for combining both of plain and custom URLs

In Wordpress, we usually set the custom URLs for the purpose of SEO.
For the old plain URLs which are previously published to other sites, we should maintain them by redirecting to the new custom URLs.

The htaccess setting can simply do the redirection to a static page, such as response 404 not found or see other new site. Sometimes, we don’t know how many old plain urls are published. It cannot fulfill our needs for mapping the old plain URLs to the corresponding new URLs. For example, we want to create the following rules of redirections:

redirect http://example.com/?p=123 to http://example.com/custom/link

redirect http://example.com/?p=456 to http://example.com/the/other_links

There may be more old URLs we don’t be aware of are published. A general soluton is to keep both of the plain and custom URLs.

In this case, we use add_filter function provided by Wordpress.
Navigate your directory of currently used theme and find functions.php and put a piece of codes as follows:

function compatible_link( $postlink, $redirect) {
    $id = get_query_var('p', -1);
    if($id !== -1 && is_numeric($id)) {
        return get_permalink($id);
    }
    return $postlink;
}

add_filter('post_type_link', 'compatible_link', 10, 2);

Done! Enjoy your website.

REFERENCE:

  1. Rewriting URLs in WordPress: Tips and Plugins. URL: http://www.hongkiat.com/blog/wordpress-url-rewrite/
  2. Wordpress的Hook機制與原理. URL: http://www.mrmu.com.tw/2011/10/10/wordpress-hook/
  3. Custom post type with post_id in permalink structure. URL: https://wordpress.org/support/topic/custom-post-type-permalink-structure
  4. Custom Login Url with plain permalink format. URL: https://wordpress.org/support/topic/custom-login-url-with-plain-permalink-format
  5. Function Reference/get query var. URL:https://codex.wordpress.org/Function_Reference/get_query_var
  6. CREATING CUSTOM PERMALINKS IN WORDPRESS. URL:http://wp-events-plugin.com/tutorials/creating-custom-permalinks-in-wordpress/

2016年4月12日 星期二

2016年2月14日 星期日

How to install SSL on CKAN

Please refer to the configuration files: https://gist.github.com/nigelbabu/076ad12c13b2071de8e1

Related references:

  1. (Chinese) Let’s Encrypt 的 SSL 憑證安裝
  2. How To Secure Nginx with Let's Encrypt on Ubuntu 14.04

2016年1月29日 星期五

[Software] Pencil Project

Web developers can try the Pencil Project. It offers not only a GUI to design the webpages, but also exports all of the webpages to a single web page. These features help you a lot to present your designed pages for customers.
Official Site: http://pencil.evolus.vn/

2016年1月14日 星期四

[eID] The short survey of electronic ID cards.

[Chinese]
根據新版台灣國民身分證的新聞報導[1],大部分人會擔心這類措施的相關問題。像是:
  1. 為什麼我們政府要遵循國際民航組織ICAO(International Civil Aviation Organization)的標準呢?
  2. 為什麼機讀碼這麼像中國政府的台胞證形式呢?
我在此做了一些關於電子身分證(electronic ID, eID)的研究:

ICAO的策略目標強烈地與United Nations Sustainable Development Goals (SDGs)做連結,滿足17項中的13項。包含了確保健康的生命、達到性別平等、建立彈性的基礎設施、確保可承受的消耗與產製樣式、降低國與國之間的不平等性等等。[2]
為了達到這些目標,其中的一個解決方案是推動可機讀旅行文件,稱為ICAO Doc 9303.[3]

為什麼ICAO Doc 9303這麼重要?因為它可讓機器容易讀取你的個人資訊,包含姓名、出生日期、出生地與其他欄位。為了保護這些隱私資料,一系列的安全措施也被採用 (請見[3] 的Part 11與Part 12)。

為了建立eID,德國的新版身分證也採用ICAO Doc 9303 [4]。(事實上,它採用其他方案 ISO/IEC standard 14443 [5],但也使用ICAO Doc 9303的部分標準[6])

這就是為何我們政府聲稱eID遵循ICAO的國際規範。

另一個問題:「為什麼機讀碼這麼像中國政府的台胞證形式呢?」
可機讀護照(Machine Readable Passport, MRP)採用可機讀區域(Machine Readable Zone)來讓機器輕易讀取[7][8],這樣的措施也被德國、義大利等國的身分證採用,新式身分證也採用類似的編碼方式。這就是為何新版台胞證有類似的機讀碼。

更多的eID趨勢,可以參見科技新報的報導[9]。

[English]
According to the news of electronic ID card for Taiwanese [1], most of people wonder the problems of this measure. Some problems arises:
  • Why does our government follow the standard of ICAO(International Civil Aviation Organization)?
  • Why is the machine readable code very similar to that of Taiwanese citizen card which China provided?
I make a short survey on the specification of electronic ID card (abbreviate eID).

ICAO's strategic objective strongly linked to 13 of the 17 United Nations Sustainable Development Goals (SDGs) proposed. Those include ensuring healthy lives, achieving gender equality, building resilient infrastructure, ensuring sustainable consumption and production patterns, reducing inequality among countries and so on.[2]
For achieving these goals, one of the solution is to promote Machine Readable Travel Documents, called ICAO Doc 9303.[3]

Why is the ICAO Doc 9303 important? Because it can let machines easily read your personal information including name, birthday, born town and other fields. For protecting the private data, a series of security measures are adopted. (See Parts 11 and 12 in [3].)

For creating eID cards, ICAO Doc 9303 is one of the solution for German new ID card [4]. (Actually, it adopts another solution called ISO/IEC standard 14443 [5] but also uses part of the specification of ICAO Doc 9303 [6]).

That's why our government say that the new ID card can be compatible with international specification - ICAO.

The other question: "Why is the machine readable code very similar to Taiwanese citizen card which China provided?"

The machine readable passport (MRP) should use a machine readable zone (MRZ) to let machine easily read the information [7][8] which is also adopted by multiple eID cards like German, Italian and so on. The Taiwanese citizen card provided by China use the similar encoding method. That's why the new Taiwanese use the MRZ for machines.
As aforementioned, the security is also ensured.
For further trend of eID, please refer to the post of TechNews.[9]

References
  1. 仝澤蓉, "新版身分證樣式曝光 將採中英對照,"  聯合財經網 [Online]. Available:  http://goo.gl/HHKvWx
  2. ICAO, "ICAO and the United Nations Sustainable Development Goals," ICAO [Online]. Available:http://www.icao.int/about-icao/aviation-development/Pages/SDG.aspx
  3. ICAO, "Machine Readable Travel Documents," ICAO [Online] Available: http://www.icao.int/publications/pages/publication.aspx?docnum=9303
  4. Moritz Horsch, Johannes Braun and Alex Wiesmaier, "Mobile eID application for the German identity card," Technische Universität Darmstadt, Technical Report [Online]. Available: https://www.cdc.informatik.tu-darmstadt.de/reports/TR/Mobile_eID_app_for_the_German_ID_card.pdf 
  5. "ISO/IEC 14443," Wikipedia [Online]. Available: https://en.wikipedia.org/wiki/ISO/IEC_14443
  6. "National identity cards in the European Economic Area," Wikipedia [Online]. Available: https://en.wikipedia.org/wiki/National_identity_cards_in_the_European_Economic_Area
  7. "Machine-readable passport," Wikipedia [Online] Available: https://en.wikipedia.org/wiki/Machine-readable_passport#Official_travel_documents 
  8. "可機讀護照," Wikiwand [Online] Available:http://goo.gl/iG87tf
  9. Sanada Yukimura, "台灣新版晶片身分證 2017 年上路? 看全球 eID 趨勢," 科技新報 [Online] Available: http://technews.tw/2015/05/11/taiwan-eid-2007/ 


2016年1月12日 星期二

[Docear4Word] The beta version for fixing bugs on displaying reference list.

Today, I downloaded the source code of Docear4Word with version 1.23 in order to fix the bug I reported in my previous post.
After struggling a while, the improvement is done.
Please visit my created repository here. (URL: https://github.com/allen501pc/Docear4Word )

2016年1月2日 星期六

[Docear4Word] Fix the bug from Docear4Word that doesn't display URL in IEEE-with-URL style.

Docear is the one of academic literature suites I love. It provides  not only a comprehensive mind map tool but also a reference management tool inherited by JabRef. By using the JabRef's managed reference results, the extension, Docear4Word , can help us easily add citations and display them with many styles. However, the provided IEEE-with-URL style has a bug on displaying URL.
No matter how URLs  you add, the generated reference list never shows URLs.
That's because the included ieee-w-url.csl doesn't add the macro for generating URLs when you use the entry types except webpage. The webpage entry type is not set as default entry types in Docear. That's why you add any entry types that Docear provided, then the URLs are never displayed!
One  solution is to replace the old ieee-w-url.csl with newer one.
According to Prof. Dr. Artur Lugmayr's post [1], we just follow these steps:
  1. Open the directory named "C:\ProgramData\Docear4Word\Styles"
  2. Getting the administrator permission, open the file called "ieee-w-url.csl" in the directory.
  3. Replace the whole content with the newer one I put in Github. (mirrored from the git [2] of citation-style-languages) The newer content of this csl is on https://github.com/allen501pc/styles/blob/master/ieee-with-url.csl. Then save it!
Docear是我喜愛的學術文獻整理套件之一。它提供了全面性的心智圖以及基於JabRef的參考文獻整理工具。其外掛 Docear4Word 藉著使用JabRef的整理文獻結果,幫助我們輕易地加入文獻參考與展現不同風格的參考文獻。然而,提供的IEEE-with-URL風格在輸出URL時,存在著一個臭蟲。
無論您如何加入URL,產生的文獻列表卻從不顯示這些URL。這是因為內建的ieee-w-url.csl 除了webpage這個entry type以外,並無產生URL的巨集。而webpage這個entry type卻沒有成為Docear的內建entry type。這就是無論你怎麼在Docear提供的entry type加入URL,這些URL卻從未顯示。
解決之道是替換掉舊的ieee-w-url.csl。根據Prof. Dr. Artur Lugmayr的貼文[1],我們只要遵循以下步驟即可:
  1. 開啟 C:\ProgramData\Docear4Word\Styles 目錄。
  2. 在此目錄中,以管理者權限開啟ieee-w-url.csl檔案。
  3. 到本人提供的Gibhubhttps://github.com/allen501pc/styles/blob/master/ieee-with-url.csl 取得該新版的內容格式,替換掉舊的(該內容格式是由Citation-Style-Languages的Github [2]鏡射而來。)

References
[1] Prof. Dr. Artur Lugmayr, "Professional and free referencing software," [Online] Available: http://artur-lugmayr.blogspot.tw/2013/09/professional-free-referencing-software.html
[2] CitationStyles.org, "Official repository for Citation Style Language (CSL) citation styles," [Online] Available: https://github.com/citation-style-language/styles

2015年11月24日 星期二

Solve the problem of Causing IIS Express Slow Down

[Chinese/中文]
最近我的Visual Studio 2013內建的IIS Express 實在跑太慢了,而且常常回報無服務回應的錯誤。根據此條目,看起來像是logger元件出問題。
為了解決此一問題,我們可以嘗試以下步驟:
1. 用系統管理員權限開啟命令提示字元視窗 (cmd console)
2. 輸入以下指令
  (1) cd  %systemroot%\system32\inetsrv\
  (2) appcmd set config /section:urlCompression /doStaticCompression:True
  (3) appcmd set config /section:urlCompression /doDynamicCompression:False
  (4) appcmd set config /section:httpLogging /dontLog:True

[English]
Recently, my Visual Studio 2013's built-in IIS Express runs too slower and it often reports errors about no service response.
According to the thread, it seems logger component of IISExpress broken.
To fix the problem, we can try the following steps:
1. Open a cmd console with Administrator permission.
2. Type the subsequent commands:
  (1) cd  %systemroot%\system32\inetsrv\
  (2) appcmd set config /section:urlCompression /doStaticCompression:True
  (3) appcmd set config /section:urlCompression /doDynamicCompression:False
  (4) appcmd set config /section:httpLogging /dontLog:True

Reference

  1. Mike Volodarsky, "Getting Start with Appcmd.exe", Microsoft Developer Network. URL: http://www.iis.net/learn/get-started/getting-started-with-iis/getting-started-with-appcmdexe
  2. "VS 2013 Professional - Web Dev Server (IIS Express) is very slow", Microsoft Developer Network. URL: https://social.msdn.microsoft.com/Forums/en-US/8e42baa5-38c7-4fef-b120-b0dd22ba8228/vs-2013-professional-web-dev-server-iis-express-is-very-slow?forum=visualstudiogeneral