Window中调试HBase问题小结

2020-12-13 05:56

阅读：358

标签：des style blog http color java 使用 os

　　1.好久没用log4j了，转到logback好多年了，hbase程序运行时，报缺少log4j配置，那么，就转去logback吧（以下的XXX表示版本号）。

　　原先lib包里面有log4j-XXX.jar、slf4j-api-XXX.jar、slf4j-log4j12-XXX.jar，干掉log4j-XXX.jar和slf4j-log4j12-XXX.jar，加入jcl-over-slf4j-XXX.jar，log4j-over-slf4j-XXX.jar、logback-classic-XXX.jar、logback-core-XXX.jar，至于原先的slf4j-api这个包，可以顺手替换成新的，然后，加入一个logback.xml到classpath下

xml version="1.0" encoding="UTF-8"?>








configuration scan="true" scanPeriod="120 seconds" debug="false">
  appender name="FileApp" class="ch.qos.logback.core.rolling.RollingFileAppender">
    
    filter class="ch.qos.logback.classic.filter.ThresholdFilter"> 
      level>WARNlevel> 
    filter>
    File>log/HBaseClient.logFile>
    encoder>
      pattern>%d [%t] %-5p %c - %m%npattern>
    encoder>
    rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      maxIndex>10maxIndex>
      FileNamePattern>log/HBaseClient.log.%iFileNamePattern>
    rollingPolicy>
    triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      MaxFileSize>1024KBMaxFileSize>
    triggeringPolicy>
  appender>
  appender name="ConApp" class="ch.qos.logback.core.ConsoleAppender">
    encoder>
      pattern>%d [%t] %-5p %c - %m%npattern>
    encoder>
  appender> 
  root level="ERROR">
      appender-ref ref="ConApp"/>
    appender-ref ref="FileApp"/>
  root>
configuration>

　　这样，log4j和系统的日志系统就被logback接管了，至于为啥不用log4j，用logback，这个。。。自己去看吧，有空我可能会写点啥，logback有点小bug，就是有的linux系统下会找不到配置文件，以前修复过，不知道新版本有处理掉没。

　　2.Could not locate executable null\bin\winutils.exe in the Hadoop binaries

　　老掉牙的问题了，系统变量设置HADOOP_HOME，我还是不想去设置环境变量，还是一行代码来得快

System.setProperty("hadoop.home.dir", "G:/hadoop/hadoop-2.4.1");

　　3.程序运行正常，但是当log级别改为Info时，突然发现了这么一条

2014-08-04 16:19:22,942 [main-SendThread(Master:2222)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server Master/192.168.117.128:2222. Will not attempt to authenticate using SASL (java.lang.SecurityException: 无法定位登录配置)

　　看到个Exception，什么情况？觉得哪里不对，但是程序又是一切正常的。

　　搜索了一通，发现好多一样的问题，但是貌似和我的没啥关系，别人是程序跑不通，我是程序正常得很。

　　有说是zookeeper版本不一致的，我看了下hadoop和hbase中的确实有区别，一个zookeeper-3.4.5.jar，一个zookeeper-3.4.6.jar，但是我总觉得不对，新旧怎么可能会不兼容啊。那就直接统一了试试，连客户端程序的都统一了，结果那个info信息还是存在，就和不换是一样的，果然不是这里的问题！

　　有说是配置和hosts的，我这里都没这问题的，都配置过了的。加参数？我在配置中都写好的，直接打印一看就知道有了。

　　这只是一个info信息，很奇怪为啥是个info，照说info级别的都是不影响啥的信息，程序确实是很正常的跑，但是看到这个信息就是不舒服。

　　自己查代码去算了，根据日志，是在ClientCnxn.java中出现的信息，好吧，查看源码

private void logStartConnect(InetSocketAddress addr) {
            String msg = "Opening socket connection to server " + addr;
            if (zooKeeperSaslClient != null) {
              msg += ". " + zooKeeperSaslClient.getConfigStatus();
            }
            LOG.info(msg);
        }

　　汗，还得换去查ZooKeeperSaslClient.java，继续吧

public ZooKeeperSaslClient(final String serverPrincipal)
            throws LoginException {
        /**
         * ZOOKEEPER-1373: allow system property to specify the JAAS
         * configuration section that the zookeeper client should use.
         * Default to "Client".
         */
        String clientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client");
        // Note that ‘Configuration‘ here refers to javax.security.auth.login.Configuration.
        AppConfigurationEntry entries[] = null;
        RuntimeException runtimeException = null;
        try {
            entries = Configuration.getConfiguration().getAppConfigurationEntry(clientSection);
        } catch (SecurityException e) {
            // handle below: might be harmless if the user doesn‘t intend to use JAAS authentication.
            runtimeException = e;
        } catch (IllegalArgumentException e) {
            // third party customized getAppConfigurationEntry could throw IllegalArgumentException when JAAS
            // configuration isn‘t set. We can reevaluate whether to catch RuntimeException instead when more 
            // different types of RuntimeException found
            runtimeException = e;
        }
        if (entries != null) {
            this.configStatus = "Will attempt to SASL-authenticate using Login Context section ‘" + clientSection + "‘";
            this.saslClient = createSaslClient(serverPrincipal, clientSection);
        } else {
            // Handle situation of clientSection‘s being null: it might simply because the client does not intend to 
            // use SASL, so not necessarily an error.
            saslState = SaslState.FAILED;
            String explicitClientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY);
            if (explicitClientSection != null) {
                // If the user explicitly overrides the default Login Context, they probably expected SASL to
                // succeed. But if we got here, SASL failed.
                if (runtimeException != null) {
                    throw new LoginException("Zookeeper client cannot authenticate using the " + explicitClientSection +
                            " section of the supplied JAAS configuration: ‘" +
                            System.getProperty(Environment.JAAS_CONF_KEY) + "‘ because of a " +
                            "RuntimeException: " + runtimeException);
                } else {
                    throw new LoginException("Client cannot SASL-authenticate because the specified JAAS configuration " +
                            "section ‘" + explicitClientSection + "‘ could not be found.");
                }
            } else {
                // The user did not override the default context. It might be that they just don‘t intend to use SASL,
                // so log at INFO, not WARN, since they don‘t expect any SASL-related information.
                String msg = "Will not attempt to authenticate using SASL ";
                if (runtimeException != null) {
                    msg += "(" + runtimeException + ")";
                } else {
                    msg += "(unknown error)";
                }
                this.configStatus = msg;
                this.isSASLConfigured = false;
            }
            if (System.getProperty(Environment.JAAS_CONF_KEY)  != null) {
                // Again, the user explicitly set something SASL-related, so they probably expected SASL to succeed.
                if (runtimeException != null) {
                    throw new LoginException("Zookeeper client cannot authenticate using the ‘" +
                            System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") +
                            "‘ section of the supplied JAAS configuration: ‘" +
                            System.getProperty(Environment.JAAS_CONF_KEY) + "‘ because of a " +
                            "RuntimeException: " + runtimeException);
                } else {
                    throw new LoginException("No JAAS configuration section named ‘" +
                            System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") +
                            "‘ was found in specified JAAS configuration file: ‘" +
                            System.getProperty(Environment.JAAS_CONF_KEY) + "‘.");
                }
            }
        }
    }

　　根据上面的代码，可知问题在于红色部分，再注意一下上面的注释，我靠，果然只是一个普通的说明文字，但是，尼玛能不能不在info信息里面搞个Exception提示出来？？？

　　好吧，那确实是一个info信息，是无关紧要的info信息，其实只是说明了。。。。。由于客户端没用SASL，所以不使用SASL进行验证。。。

　　能不能换个好点的说明文字？我这些操作都是多余的了！人家就那样显示的，唉。看得不爽也没办法了，难道为了这点东西还去改源码不成？够汗的。　　

　　解决办法，客户端添加SASL信息，然后服务端也改掉，添加验证，这个官方文档http://hbase.apache.org/book.html#security有说明了，直接看去就是了，这里不说明了。

　　唉，处女座追求完美的心，果然是不变的。。。加上，看不到烦心的文字，舒坦了。。

Window中调试HBase问题小结,搜素材,soscw.com

Window中调试HBase问题小结

标签：des style blog http color java 使用 os

原文地址：http://www.cnblogs.com/zxub/p/3890641.html

上一篇：优化网站该丰富的10项内容

下一篇：python3 与dict相关的魔法方法。使用于二叉搜索树的类中

文章来自：搜素材网的编程语言模块，转载请注明文章出处。
文章标题：Window中调试HBase问题小结
文章链接：http://soscw.com/essay/32062.html

亲，登录后才可以留言！

Window中调试HBase问题小结

评论

热门文章

推荐文章

最新文章

置顶文章