Process SequenceFile without Enabling Hadoop Platform

Recently I got a requirement for reading Hadoop’s SequenceFile without enabling Hadoop Platform. However, most examples introduce the read/write SequenceFile with Hadoop Platform. How do I read such files without hadoop?
There’s a tricky solution in this case.
1. Download Hadoop binary file from hadoop site. For Linux/Unix please directly download it; for Windows, there’s pre-built archive file – hadoop-common-2.2.0-bin (source code is here) , created by Abhijit Ghosh.
2. Set environment variable HADOOP_HOME by the directory path (suppose the directory is /usr/local/hadoop in Unix ; or C:/hadoop-common-2.2.0-bin in Windows)
3. Append $HADOOP_HOME/bin to the end of environment variable PATH. ( i.e. /usr/local/hadoop/bin in Unix ; or C:/hadoop-common-2.2.0-bin/bin in Windows
4. Write your program like this (Notice that you have to download hadoop-common-2.2+):


import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.util.ReflectionUtils;

public class ProcessSequenceFile {

public static void readSequenceFile(String sequenceFileName) throws IOException, URISyntaxException {
Configuration conf = new Configuration();
String directoryPath = "file:///";
directoryPath = URLEncoder.encode(directoryPath, "UTF-8");
FileSystem fs = FileSystem.get(new URI(directoryPath),conf);
Path file = new Path(fs.getUri().toString() + sequenceFileName);
SequenceFile.Reader reader = new SequenceFile.Reader(fs, file, conf);
Text key = (Text) ReflectionUtils.newInstance(reader.getKeyClass(), conf);
Text value = (Text) ReflectionUtils.newInstance(reader.getValueClass(), conf);
while(,value)) {
System.out.println("Key:" + key);



public static void main(String[] args) {
try {
// If args[0] is the SequenceFile we need to read.
} catch (IOException | URISyntaxException e) {
// TODO Auto-generated catch block

Enjoy it!

1. Gnosis Runmination, “WIN7下运行hadoop程序报:Failed to locate the winutils binary in the hadoop binary path.” Available: [Online]
2. StackOverFlow, “Running Apache Hadoop 2.1.0 on Windows”. Available: [Online]
3.  Abhijit Ghosh, “ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path,” Available: [Online]
4. Hadoop, “Native Libraries Guide.” Available: [Online]

This entry was posted in hadoop, Java, Programming, 程式設計. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *