hadoop入门-在windows上编译x64位hadoop

标签：hadoop   windows   visual studio 2010   源码   

环境

Windows 7 x64位，visual studio 2010 professional

hadoop 源码版本2.2.0

步骤（来源于书《Pro Apache Hadoop, Second Edition》略作修改。

1. Ensure that JDK 1.6 is or higher is installed. We assume that it is installed in the c:/MyApps/jdkl6/ folder, which should have a bin subfolder.

2. Download the hadoop-2.2.x-src.tar.gz files (2.2.0 at the time of this writing) from the download section of the Apache web site for Hadoop. The download link is as follows: http://www.apache.org/dist/hadoop/core/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz.

3. Explode the tar file into a directory. For the purpose of this section, we assume that the directory is called c:/myapps/hadoop/.

4. If using Visual Studio, use Visual Studio 2010 Professional (not 2012). Do not use Visual Studio Express because it does not support compiling for 64-bit, which presents problems if running on a 64-bit system. Alternatively, download the Microsoft Windows SDK v7.1. At the time of writing, the link is the following:

   http://www.microsoft.com/en-us/download/confirmation.aspx?id=8279.

5. Place the SDK in the folder with full privileges. For this section, we assume that the folder is c:/myapps/winsdk71/.

6. Download and install the Linux-like environment for Windows named Cygwin. We assume that Cygwin home folder is c:/myapps/cygwin/. Ensure that the following UNIX command-line tools are installed: sh, mkdir, rm, cp, tar, and gzip. The download location for Cygwin is http://cygwin.com/install.html.

7. Download and install Maven 3.1.1 or a compatible version. Installation consists of simply exploding the tar bundle into a Windows folder. We assume that the home folder for Maven download is c:/MyApps/apache-maven-3.1.1/. Verify that this folder has the bin subfolder.

8. Download and install Protocol Buffers 2.5.0. At the time of writing, the link is http://protobuf.googlecode.com/files/protoc-2.5.0-win32.zip.

9. Simply unzip the zip file into a folder (assumed to be c:/MyApps/protobuf/). Verify that the installation is correct by ensuring that the folder contains the protoc.exe file.

10. Install ZLIB from http://zlib.net/zlib128-dll.zip and explode the zip file in the c:/zlib-1.2.8 folder. At runtime, this folder must be accessible from the PATH variable.

11. Add the environment variables shown in Table A-1.

Table A-1: Environment Variables for Windows

Environment Variable	Value
JAVA_HOME	c:/MyApps/jdk16/
M2_HOME	c:/MyApps/apache-maven-3.1.1/
Platform	x64 (or Win32 when building on the 32-bit system)
CYGWIN_HOME	c:/myapps/cygwin/
PROTOBUF_HOME	c:/myapps/protobuf/
ZLIB_HOME	c:/zlib-1.2.8 12 .Add the following to the Windows PATH variable: %JAVA_HOME%/bin/ %CYGWIN_HOME%/bin/ %M2_HOME%/bin/ %PROTOBUF_HOME% %ZLIB_HOME

开始编译，执行下面的步骤：

hadoop 2.2.0在编译的时候有一个bug，参考https://issues.apache.org/jira/browse/HADOOP-10110描述，然后下载https://issues.apache.org/jira/secure/attachment/12614482/HADOOP-10110.patch这个patch，按照这个patch修改hadoop-common-project/hadoop-auth/pom.xml文件，然后再进行编译，否则会出错。

1. Go to the Windows Start  All Programs  Microsoft Windows SDK v7.1 and start the Windows 
   SDK 7.1 command prompt in Run As Administrator mode.
   

3. Change the folder to c:/MyApps/hadoop. Recall that this is the folder in which the Hadoop source was placed.

4. Execute the following command mvn package with the -Pdist,native-win -DskipTests -Dtar options. The exact command is this:

   
   mvn package -Pdist,native-win -DskipTests –Dtar
   

5. If there are no errors, the hadoop-2.2.0.tar.gz file is created in the c:/myapps/hadoop/ hadoop-dist/hadoop-2.2.0/ folder.

编译完后，如果没有任何错误，在hadoop-2.2.0-src\hadoop-dist\target目录下面会生成hadoop-2.2.0.tar.gz文件。

第一步大功告成！

hadoop入门-在windows上编译x64位hadoop

标签：hadoop   windows   visual studio 2010   源码   

原文地址：http://blog.csdn.net/zhanghaiyang9999/article/details/40794235