在CentOS6.9安装Azkaban3.50.0_centos6.9 azkaban-程序员宅基地
Azkaban简介
上图是Azkaban官网提供的架构图，包括三个组件：
Relational Database (MySQL)
AzkabanWebServer
AzkabanExecutorServer
组件分工：
元数据库：用于维护任务流信息以及执行情况信息。
Web服务器：用户交互的门户，同时也是管理任务流、工程，控制调度，任务分配的核心组件。
执行服务器：任务被真正调起的服务器，可扩展。
AzkabanWebServer对数据库的使用：
项目管理
追踪任务流的状态
查看任务执行结果以及历史日志
调度任务
sla
AzkabanExecutorServer 对数据库的使用：
从数据库获取工程的信息
从数据库获取要执行的任务流
储存任务流和任务的日志信息
从数据库获取内置任务流的任务依赖关系
一、环境
操作系统：CentOS6.9
软件版本：Azkaban3.50.0
二、安装
我这里是将Azkaban安装在hadoop用户，所以所有的linux操作都在hadoop用户。
1、编译Azkaban可执行文件
1.1 下载安装包
cd /data/install_pkg
wget https://github.com/azkaban/azkaban/archive/3.50.0.tar.gz
解压安装包：
tar -zxvf 3.50.0.tar.gz
1.2 修改邮件代码
vi /data/install_pkg/azkaban-3.50.0/azkaban-common/src/main/java/azkaban/utils/EmailMessage.java
找到邮件参数配置的地方
:/props
增加以下语句：
props.put("mail.smtp.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
1.3 安装git
因为使用gradle编译会用到git，所以要安装：
yum install git
1.4 安装JCE
如果不安装JCE，后面编译的时候会报错。安装方法见：
https://blog.csdn.net/andyguan01_2/article/details/88343641
1.5 编译文件
使用Gradle编译文件。Gradle是一个基于Apache Ant和Apache Maven的项目自动化构建工具。
cd /data/install_pkg/azkaban-3.50.0
./gradlew build installDist
编译报错，看着像是测试过程中出错，没找具体到是什么原因。
Starting a Gradle Daemon (subsequent builds will be faster)
Parallel execution with configuration on demand is an incubating feature.
> Task :az-core:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-common:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-web-server:npmSetup
/data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/bin/npm -> /data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/bin/npm-cli.js
/data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/bin/npx -> /data/install_pkg/azkaban-3.50.0/azkaban-web-server/.gradle/npm/npm-v5.6.0/lib/node_modules/npm/bin/npx-cli.js
+ [email protected]
added 476 packages in 14.414s
> Task :azkaban-exec-server:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-common:compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-hadoop-security-plugin:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /data/install_pkg/azkaban-3.50.0/azkaban-hadoop-security-plugin/src/main/java/azkaban/security/HadoopSecurityManager_H_2_0.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-common:test
azkaban.utils.UtilsTest > testValidCronExpressionV FAILED
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)EmailMessageCreatorTest
at org.junit.Assert.assertTrue(Assert.java:52)
at azkaban.utils.UtilsTest.testValidCronExpressionV(UtilsTest.java:63)
> Task :azkaban-web-server:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :azkaban-exec-server:compileTestJava
Note: /data/install_pkg/azkaban-3.50.0/azkaban-exec-server/src/test/java/azkaban/execapp/AzkabanExecutorServerTest.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :azkaban-web-server:npm_install
added 39 packages in 3.23s
> Task :azkaban-common:test
328 tests completed, 1 failed, 18 skipped
> Task :azkaban-solo-server:compileJava
Note: /data/install_pkg/azkaban-3.50.0/azkaban-solo-server/src/main/java/azkaban/soloserver/AzkabanSingleServer.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :az-hdfs-viewer:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :az-reportal:compileJava
Note: /data/install_pkg/azkaban-3.50.0/az-reportal/src/main/java/azkaban/reportal/util/StreamProviderHDFS.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :az-hadoop-jobtype-plugin:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /data/install_pkg/azkaban-3.50.0/az-hadoop-jobtype-plugin/src/main/java/azkaban/jobtype/HadoopSecureSparkWrapper.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':azkaban-common:test'.
> There were failing tests. See the report at: file:///data/install_pkg/azkaban-3.50.0/azkaban-common/build/reports/tests/test/index.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2m 4s
71 actionable tasks: 66 executed, 5 from cache
于是改为使用命令./gradlew build installDist -x test，-x test表示跳过测试。
./gradlew clean
./gradlew build installDist -x test
1.6 复制可执行文件到安装目录
编译完成后，我们就可以获取到web，exec，db，solo的可执行文件tar包，拷贝到Azkaban安装目录：
ll /data/install_pkg/azkaban-3.50.0/azkaban-*/build/distributions/*.tar.gz
cp /data/install_pkg/azkaban-3.50.0/azkaban-*/build/distributions/*.tar.gz /u01/app/azkaban-3.50.0
ll /u01/app/azkaban-3.50.0
解压tar包到Azkaban安装目录：
cd /u01/app/azkaban-3.50.0
tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
tar -zxvf azkaban-solo-server-0.1.0-SNAPSHOT.tar.gz -C /u01/app/azkaban-3.50.0
2、初始化MySQL
初始化MySQL的库、用户、权限、表、数据：
mysql -u root -p
CREATE DATABASE azkaban;
CREATE USER 'azkaban'@'%' IDENTIFIED BY '填实际密码';
GRANT SELECT,INSERT,UPDATE,DELETE ON azkaban.* to 'azkaban'@'%' WITH GRANT OPTION;
flush privileges;
use azkaban;
source /u01/app/azkaban-3.50.0/azkaban-db-0.1.0-SNAPSHOT/create-all-sql-0.1.0-SNAPSHOT.sql;
3、配置SSL
keytool -keystore keystore -alias jetty -genkey -keyalg RSA
要输入密码的地方，都填相同密码。几个问题的回答，what … country code … ?填CN，Is … correct?填y，其他问题可以直接回车不填。。
完成上述工作后，将在当前目录生成 keystore 证书文件，将keystore移动到azkaban web服务器根目录中。
mv keystore /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT
4、配置web-server
因为web-server和exec-server都没有默认配置文件，我们需要复制solo的配置文件。
cd /u01/app/azkaban-3.50.0
cp -r azkaban-solo-server-0.1.0-SNAPSHOT/conf azkaban-web-server-0.1.0-SNAPSHOT
cp /data/install_pkg/azkaban-3.50.0/azkaban-web-server/src/test/resources/log4j.properties azkaban-web-server-0.1.0-SNAPSHOT/conf
4.1 配置azkaban.properties
vi /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/azkaban.properties
配置以下内容：
# Azkaban Personalization Settings
#服务器UI名称,用于服务器上方显示的名字
azkaban.name=Test
#描述
azkaban.label=My Local Azkaban
#UI颜色
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
#默认根web目录建议最好写绝对路径，以免出现找不到文件错误
web.resource.dir=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/web
#默认时区，已改为亚洲/上海，默认为美国
default.timezone.id=Asia/Shanghai
# Azkaban UserManager class
#用户权限管理默认类
user.manager.class=azkaban.user.XmlUserManager
#用户配置文件所在位置
user.manager.xml.file=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/azkaban-users.xml
# Loader for projects
# global配置文件所在位置
executor.global.properties=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/global.properties
azkaban.project.dir=projects
#数据库类型
database.type=mysql
#端口号
mysql.port=3306
#数据库连接IP
mysql.host=填实际IP
#数据库名
mysql.database=azkaban
#数据库用户名
mysql.user=azkaban
#数据库密码
mysql.password=填实际密码
#最大连接数
mysql.numconnections=100
# Azkaban Jetty server properties.
# Jetty服务器属性.
velocity.dev.mode=false
#最大线程数
jetty.maxThreads=25
#Jetty SSL端口
jetty.ssl.port=8443
#Jetty端口
jetty.port=8081
#SSL文件名
jetty.keystore=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/keystore
#SSL文件密码
jetty.password=填实际密码
#Jetty主密码 与 keystore文件相同
jetty.keypassword=填实际密码
#SSL文件名
jetty.truststore=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/keystore
# SSL文件密码
jetty.trustpassword=填实际密码
#jetty.use.ssl=false
jetty.maxThreads=25
# Azkaban Executor settings
executor.port=12321
# mail settings
mail.sender=
mail.host=
# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy
between Azkaban web servers and users.
# enduser -> myazkabanhost:443 -> proxy -> localhost:8081
# when this parameters set then these parameters are used to generate email links.
# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.
# azkaban.webserver.external_hostname=myazkabanhost.com
# azkaban.webserver.external_ssl_port=443
# azkaban.webserver.external_port=8081
job.failure.email=
job.success.email=
lockdown.create.projects=false
cache.directory=cache
# JMX stats
jetty.connector.stats=true
executor.connector.stats=true
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes
4.2 配置commonprivate.properties
先创建配置文件：
mkdir -p /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes
vi /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes/commonprivate.properties
输入以下内容：
azkaban.native.lib=false
execute.as.user=false
4.3 配置azkaban-users.xml
修改用户配置文件azkaban-users.xml：
vi /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/conf/azkaban-users.xml
增加admin用户：（在第4行）
<azkaban-users>
<user groups="azkaban" password="填实际密码" roles="admin" username="azkaban"/>
<user password="填实际密码" roles="metrics" username="metrics"/>
<user username="admin" password="填实际密码" roles="admin,metrics"/>
<role name="admin" permissions="ADMIN"/>
<role name="metrics" permissions="METRICS"/>
</azkaban-users>
5、配置executor
将 azkaban-web-server-0.1.0-SNAPSHOT目录下的conf拷贝到 azkaban-exec-server-0.1.0-SNAPSHOT
cd /u01/app/azkaban-3.50.0
cp -r azkaban-web-server-0.1.0-SNAPSHOT/conf azkaban-exec-server-0.1.0-SNAPSHOT
cp -r azkaban-web-server-0.1.0-SNAPSHOT/plugins azkaban-exec-server-0.1.0-SNAPSHOT
6、启动服务
6.1 启动Executor Server
进入azkaban-exec-server-0.1.0-SNAPSHOT下启动（注意不要进入bin目录启动）
cd /u01/app/azkaban-3.50.0/azkaban-exec-server-0.1.0-SNAPSHOT
./bin/start-exec.sh
会在当前目录生成.out的日志文件：
ll /u01/app/azkaban-3.50.0/azkaban-exec-server-0.1.0-SNAPSHOT/*.out
more /u01/app/azkaban-3.50.0/azkaban-exec-server-0.1.0-SNAPSHOT/*.out
日志内容为：
Using Hadoop from /u01/app/hadoop
Using Hive from /u01/app/apache-hive-2.3.4-bin
./bin/internal/../..
.:/usr/java/jdk1.8.0_121/lib/dt.jar:/usr/java/jdk1.8.0_121/lib/tools.jar:/u01/app/hadoop-2.6.5/etc/hadoop:/u01/app/hadoop/share/hado
op/common/lib/*:/u01/app/hadoop/share/hadoop/common/*:/u01/app/hadoop/share/hadoop/hdfs:/u01/app/hadoop/share/hadoop/hdfs/lib/*:/u01
/app/hadoop/share/hadoop/hdfs/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/lib/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/*:/u01/app/had
oop/share/hadoop/mapreduce/lib/*:/u01/app/hadoop/share/hadoop/mapreduce/*:/u01/app/hadoop/contrib/capacity-scheduler/*.jar:./bin/int
ernal/../../lib/activation-1.1.jar:./bin/internal/../../lib/aopalliance-1.0.jar:./bin/internal/../../lib/az-core-0.1.0-SNAPSHOT.jar:
./bin/internal/../../lib/azkaban-common-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-db-0.1.0-SNAPSHOT.jar:./bin/internal/../
../lib/azkaban-exec-server-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-hadoop-security-plugin-0.1.0-SNAPSHOT.jar:./bin/inter
nal/../../lib/azkaban-spi-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/c3p0-0.9.1.1.jar:./bin/internal/../../lib/commons-codec-1.9.ja
r:./bin/internal/../../lib/commons-collections-3.2.2.jar:./bin/internal/../../lib/commons-dbcp2-2.1.1.jar:./bin/internal/../../lib/c
ommons-dbutils-1.5.jar:./bin/internal/../../lib/commons-fileupload-1.2.1.jar:./bin/internal/../../lib/commons-io-2.4.jar:./bin/inter
nal/../../lib/commons-jexl-2.1.1.jar:./bin/internal/../../lib/commons-lang-2.6.jar:./bin/internal/../../lib/commons-logging-1.2.jar:
./bin/internal/../../lib/commons-math3-3.0.jar:./bin/internal/../../lib/commons-pool2-2.4.2.jar:./bin/internal/../../lib/gson-2.8.1.
jar:./bin/internal/../../lib/guava-21.0.jar:./bin/internal/../../lib/guice-4.1.0.jar:./bin/internal/../../lib/httpclient-4.5.3.jar:.
/bin/internal/../../lib/httpcore-4.4.6.jar:./bin/internal/../../lib/jackson-core-asl-1.9.5.jar:./bin/internal/../../lib/jackson-mapp
er-asl-1.9.5.jar:./bin/internal/../../lib/javax.inject-1.jar:./bin/internal/../../lib/jetty-6.1.26.jar:./bin/internal/../../lib/jett
y-util-6.1.26.jar:./bin/internal/../../lib/joda-time-2.0.jar:./bin/internal/../../lib/jopt-simple-4.3.jar:./bin/internal/../../lib/j
sr305-3.0.2.jar:./bin/internal/../../lib/kafka-clients-0.10.0.0.jar:./bin/internal/../../lib/kafka-log4j-appender-0.10.0.0.jar:./bin
/internal/../../lib/log4j-1.2.17.jar:./bin/internal/../../lib/lz4-1.3.0.jar:./bin/internal/../../lib/mail-1.4.5.jar:./bin/internal/.
./../lib/metrics-core-3.1.0.jar:./bin/internal/../../lib/metrics-jvm-3.1.0.jar:./bin/internal/../../lib/mysql-connector-java-5.1.28.
jar:./bin/internal/../../lib/quartz-2.2.1.jar:./bin/internal/../../lib/servlet-api-2.5-20081211.jar:./bin/internal/../../lib/slf4j-a
pi-1.7.21.jar:./bin/internal/../../lib/slf4j-log4j12-1.7.21.jar:./bin/internal/../../lib/snakeyaml-1.18.jar:./bin/internal/../../lib
/snappy-java-1.1.2.4.jar:./bin/internal/../../lib/velocity-1.7.jar:./bin/internal/../../extlib/*.jar:./bin/internal/../../plugins/*/
*.jar:/u01/app/hadoop/conf:/u01/app/hadoop/*:/u01/app/apache-hive-2.3.4-bin/conf:/u01/app/apache-hive-2.3.4-bin/lib/*
Starting AzkabanExecutorServer on port 12321 ...
2019/03/08 15:01:08.059 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/03/08 15:01:08.084 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./bin/internal/../../conf
2019/03/08 15:01:08.084 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/03/08 15:01:08.094 +0800 ERROR [StdOutErrRedirect] SLF4J: Class path contains multiple SLF4J bindings.
2019/03/08 15:01:08.094 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/hadoop-2.6.5/share/hadoop/common/
lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:01:08.095 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/azkaban-3.50.0/azkaban-exec-serve
r-0.1.0-SNAPSHOT/lib/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:01:08.095 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/apache-hive-2.3.4-bin/lib/log4j-s
lf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:01:08.095 +0800 ERROR [StdOutErrRedirect] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanat
ion.
2019/03/08 15:01:08.097 +0800 ERROR [StdOutErrRedirect] SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2019/03/08 15:01:08.607 +0800 WARN [StorageCleaner] Project cleanup disabled. All artifacts will be stored.
2019/03/08 15:01:08.608 +0800 INFO [AzkabanExecServerModule] Event reporter is not enabled
2019/03/08 15:01:08.608 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/03/08 15:01:08.609 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/03/08 15:01:08.611 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:01:08.611 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/03/08 15:01:08.612 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/03/08 15:01:08.613 +0800 INFO [JobTypeManager] Loading plugin default job types
2019/03/08 15:01:08.616 +0800 INFO [JobTypeManager] Job type plugin directory set. Loading extra job types from /u01/app/azkaban-3.5
0.0/azkaban-web-server-0.1.0-SNAPSHOT/plugins/jobtypes
2019/03/08 15:01:08.616 +0800 INFO [JobTypeManager] Common plugin job props file /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SN
APSHOT/plugins/jobtypes/common.properties not found. Using only globals props
2019/03/08 15:01:08.616 +0800 INFO [JobTypeManager] Common plugin load props file /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-S
NAPSHOT/plugins/jobtypes/commonprivate.properties found. Attempt to load.
2019/03/08 15:01:08.626 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/03/08 15:01:08.628 +0800 INFO [ExecJettyServerModule] Setting up connector with stats on: true
2019/03/08 15:01:08.628 +0800 INFO [ExecJettyServerModule] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/03/08 15:01:08.628 +0800 INFO [ExecJettyServerModule] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/03/08 15:01:08.663 +0800 INFO [JmxJobMBeanManager] Initializing azkaban.execapp.jmx.JmxJobMBeanManager
2019/03/08 15:01:08.663 +0800 INFO [AzkabanExecutorServer] Job callback enabled? true
2019/03/08 15:01:08.665 +0800 INFO [JobCallbackManager] Initializing
2019/03/08 15:01:08.667 +0800 INFO [JobCallbackRequestMaker] responseWaitTimeoutMS: 5000
2019/03/08 15:01:08.668 +0800 INFO [JobCallbackRequestMaker] Global request configuration [expectContinueEnabled=false, proxy=null,
localAddress=null, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=
false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=3000, c
onnectTimeout=3000, socketTimeout=3000, contentCompressionEnabled=true]
2019/03/08 15:01:08.942 +0800 INFO [JobCallbackRequestMaker] Jobcall thread pool size: 10
2019/03/08 15:01:08.943 +0800 INFO [JobCallbackRequestMaker] Initialization for azkaban.execapp.event.JobCallbackRequestMaker is com
pleted
2019/03/08 15:01:08.944 +0800 INFO [JobCallbackManager] Initialization completed azkaban.execapp.event.JobCallbackManager
2019/03/08 15:01:08.944 +0800 INFO [JobCallbackManager] azkabanHostName oracle02.auditonline.prd.df.cn
2019/03/08 15:01:08.945 +0800 INFO [AzkabanExecutorServer] Registering MBeans...
2019/03/08 15:01:08.953 +0800 INFO [AzkabanExecutorServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/03/08 15:01:08.956 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2019/03/08 15:01:08.957 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2019/03/08 15:01:08.958 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2019/03/08 15:01:08.958 +0800 INFO [AzkabanExecutorServer] No value for property: jmx.attribute.processor.class was found
2019/03/08 15:01:08.958 +0800 INFO [log] jetty-6.1.26
2019/03/08 15:01:08.992 +0800 INFO [log] Started [email protected]:12321
2019/03/08 15:01:09.792 +0800 INFO [AzkabanExecutorServer] Started Executor Server on oracle02.auditonline.prd.df.cn:12321
2019/03/08 15:01:09.831 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/03/08 15:02:08.614 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:03:08.614 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:03:08.615 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/03/08 15:04:08.615 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:05:08.615 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:05:08.616 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/03/08 15:06:08.616 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:07:08.616 +0800 INFO [FlowRunnerManager] # of executing flows: 0
2019/03/08 15:07:08.617 +0800 INFO [FlowRunnerManager] Cleaning recently finished
查看JPS，会出现AzkabanExecutorServer：
jps
6.2 启动Web Server
进入azkaban-web-server-0.1.0-SNAPSHOT下启动（注意不要进入bin目录启动）
cd /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT
./bin/start-web.sh
会在当前目录生成.out的日志文件：
ll /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/*.out
more /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/*.out
我这边启动后报错：
java.lang.NoSuchMethodError:com.google.common.collect.ImmutableMap.toImmutableMap
解决方法见：
https://blog.csdn.net/andyguan01_2/article/details/88353889
问题解决后，重新启动Web Server，日志内容为：
Using Hadoop from /u01/app/hadoop
Using Hive from /u01/app/apache-hive-2.3.4-bin
./bin/internal/../..
.:/usr/java/jdk1.8.0_121/lib/dt.jar:/usr/java/jdk1.8.0_121/lib/tools.jar:/u01/app/hadoop-2.6.5/etc/hadoop:/u01/app/hadoop/share/hado
op/common/lib/*:/u01/app/hadoop/share/hadoop/common/*:/u01/app/hadoop/share/hadoop/hdfs:/u01/app/hadoop/share/hadoop/hdfs/lib/*:/u01
/app/hadoop/share/hadoop/hdfs/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/lib/*:/u01/app/hadoop-2.6.5/share/hadoop/yarn/*:/u01/app/had
oop/share/hadoop/mapreduce/lib/*:/u01/app/hadoop/share/hadoop/mapreduce/*:/u01/app/hadoop/contrib/capacity-scheduler/*.jar:./bin/int
ernal/../../lib/activation-1.1.jar:./bin/internal/../../lib/antlr-2.7.2.jar:./bin/internal/../../lib/aopalliance-1.0.jar:./bin/inter
nal/../../lib/az-core-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/az-flow-trigger-dependency-plugin-0.1.0-SNAPSHOT.jar:./bin/interna
l/../../lib/azkaban-common-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-db-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaba
n-spi-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/azkaban-web-server-0.1.0-SNAPSHOT.jar:./bin/internal/../../lib/c3p0-0.9.1.1.jar:./
bin/internal/../../lib/cglib-nodep-2.2.jar:./bin/internal/../../lib/commons-beanutils-1.7.0.jar:./bin/internal/../../lib/commons-cha
in-1.1.jar:./bin/internal/../../lib/commons-codec-1.9.jar:./bin/internal/../../lib/commons-collections-3.2.2.jar:./bin/internal/../.
./lib/commons-compress-1.2.jar:./bin/internal/../../lib/commons-dbcp2-2.1.1.jar:./bin/internal/../../lib/commons-dbutils-1.5.jar:./b
in/internal/../../lib/commons-digester-1.8.jar:./bin/internal/../../lib/commons-fileupload-1.2.1.jar:./bin/internal/../../lib/common
s-io-2.4.jar:./bin/internal/../../lib/commons-jexl-2.1.1.jar:./bin/internal/../../lib/commons-lang-2.6.jar:./bin/internal/../../lib/
commons-logging-1.2.jar:./bin/internal/../../lib/commons-math3-3.0.jar:./bin/internal/../../lib/commons-pool2-2.4.2.jar:./bin/intern
al/../../lib/commons-validator-1.3.1.jar:./bin/internal/../../lib/data-1.15.7.jar:./bin/internal/../../lib/data-transform-1.15.7.jar
:./bin/internal/../../lib/dom4j-1.1.jar:./bin/internal/../../lib/gson-2.8.1.jar:./bin/internal/../../lib/guava-21.0.jar:./bin/intern
al/../../lib/guice-4.1.0.jar:./bin/internal/../../lib/httpclient-4.5.3.jar:./bin/internal/../../lib/httpcore-4.4.6.jar:./bin/interna
l/../../lib/jackson-core-2.2.2.jar:./bin/internal/../../lib/jackson-core-asl-1.9.5.jar:./bin/internal/../../lib/jackson-mapper-asl-1
.9.5.jar:./bin/internal/../../lib/javax.inject-1.jar:./bin/internal/../../lib/javax.servlet-api-3.0.1.jar:./bin/internal/../../lib/j
etty-6.1.26.jar:./bin/internal/../../lib/jetty-util-6.1.26.jar:./bin/internal/../../lib/joda-time-2.0.jar:./bin/internal/../../lib/j
opt-simple-4.3.jar:./bin/internal/../../lib/json-20070829.jar:./bin/internal/../../lib/li-jersey-uri-1.15.7.jar:./bin/internal/../..
/lib/log4j-1.2.17.jar:./bin/internal/../../lib/mail-1.4.5.jar:./bin/internal/../../lib/metrics-core-3.1.0.jar:./bin/internal/../../l
ib/metrics-jvm-3.1.0.jar:./bin/internal/../../lib/mina-core-1.1.7.jar:./bin/internal/../../lib/mysql-connector-java-5.1.28.jar:./bin
/internal/../../lib/netty-3.2.3.Final.jar:./bin/internal/../../lib/oro-2.0.8.jar:./bin/internal/../../lib/parseq-1.3.6.jar:./bin/int
ernal/../../lib/pegasus-common-1.15.7.jar:./bin/internal/../../lib/quartz-2.2.1.jar:./bin/internal/../../lib/r2-1.15.7.jar:./bin/int
ernal/../../lib/restli-common-1.15.7.jar:./bin/internal/../../lib/restli-server-1.15.7.jar:./bin/internal/../../lib/servlet-api-2.5-
20081211.jar:./bin/internal/../../lib/slf4j-api-1.7.18.jar:./bin/internal/../../lib/slf4j-log4j12-1.7.18.jar:./bin/internal/../../li
b/snakeyaml-1.18.jar:./bin/internal/../../lib/snappy-0.3.jar:./bin/internal/../../lib/sslext-1.2-0.jar:./bin/internal/../../lib/stru
ts-core-1.3.8.jar:./bin/internal/../../lib/struts-taglib-1.3.8.jar:./bin/internal/../../lib/struts-tiles-1.3.8.jar:./bin/internal/..
/../lib/velocity-1.7.jar:./bin/internal/../../lib/velocity-tools-2.0.jar:./bin/internal/../../extlib/*.jar:./bin/internal/../../plug
ins/*/*.jar:/u01/app/hadoop/conf:/u01/app/hadoop/*:/u01/app/apache-hive-2.3.4-bin/conf:/u01/app/apache-hive-2.3.4-bin/lib/*
2019/03/08 15:02:55.688 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/03/08 15:02:55.708 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./bin/internal/../../conf
2019/03/08 15:02:55.709 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/03/08 15:02:55.718 +0800 ERROR [StdOutErrRedirect] SLF4J: Class path contains multiple SLF4J bindings.
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/hadoop-2.6.5/share/hadoop/common/
lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/azkaban-3.50.0/azkaban-web-server
-0.1.0-SNAPSHOT/lib/slf4j-log4j12-1.7.18.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: Found binding in [jar:file:/u01/app/apache-hive-2.3.4-bin/lib/log4j-s
lf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2019/03/08 15:02:55.719 +0800 ERROR [StdOutErrRedirect] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanat
ion.
2019/03/08 15:02:55.722 +0800 ERROR [StdOutErrRedirect] SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2019/03/08 15:02:56.157 +0800 INFO [WebServerProvider] Excluded Cipher Suites: []
2019/03/08 15:02:56.161 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/03/08 15:02:56.161 +0800 INFO [WebServerProvider] Setting up connector with stats on: true
2019/03/08 15:02:56.163 +0800 INFO [WebServerProvider] Starting SSL server on port: 8443 # Max threads: 25
2019/03/08 15:02:56.679 +0800 INFO [ExecutorManager] Initializing local executor localhost:12321
2019/03/08 15:02:57.044 +0800 WARN [StorageCleaner] Project cleanup disabled. All artifacts will be stored.
2019/03/08 15:02:57.044 +0800 INFO [AzkabanProjectLoader] Using temp dir: /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-SNAPSHOT/
temp
2019/03/08 15:02:57.044 +0800 INFO [AzkabanProjectLoader] Project version retention is set to 3
2019/03/08 15:02:57.046 +0800 WARN [XmlValidatorManager] Validator directory validators does not exist or is not a directory.
2019/03/08 15:02:57.046 +0800 WARN [XmlValidatorManager] Azkaban properties file does not contain the key project.validators.xml.fil
e
2019/03/08 15:02:57.103 +0800 INFO [TriggerManager] TriggerManager loaded.
2019/03/08 15:02:57.125 +0800 INFO [AzkabanWebServerModule] Loading user manager class azkaban.user.XmlUserManager
2019/03/08 15:02:57.233 +0800 INFO [XmlUserManager] Loading user azkaban
2019/03/08 15:02:57.233 +0800 INFO [XmlUserManager] Loading user metrics
2019/03/08 15:02:57.234 +0800 INFO [XmlUserManager] Loading user admin
2019/03/08 15:02:57.287 +0800 INFO [AzkabanWebServer] Loading built-in checker and action types
2019/03/08 15:02:57.290 +0800 INFO [CheckerTypeLoader] Registering checker BasicTimeChecker
2019/03/08 15:02:57.291 +0800 INFO [CheckerTypeLoader] Registering checker SlaChecker
2019/03/08 15:02:57.291 +0800 INFO [CheckerTypeLoader] Registering checker ExecutionChecker
2019/03/08 15:02:57.291 +0800 INFO [ActionTypeLoader] Registering action ExecuteFlowAction
2019/03/08 15:02:57.291 +0800 INFO [ActionTypeLoader] Registering action KillExecutionAction
2019/03/08 15:02:57.292 +0800 INFO [ActionTypeLoader] Registering action AlertAction
2019/03/08 15:02:57.292 +0800 INFO [ActionTypeLoader] Registering action CreateTriggerAction
2019/03/08 15:02:57.293 +0800 INFO [PluginCheckerAndActionsLoader] Loading plug-in checker and action types
2019/03/08 15:02:57.293 +0800 ERROR [PluginCheckerAndActionsLoader] plugin path plugins/triggers doesn't exist!
2019/03/08 15:02:57.314 +0800 INFO [AzkabanWebServer] Setting timezone to Asia/Shanghai
2019/03/08 15:02:57.314 +0800 INFO [AzkabanWebServer] Registering MBeans...
2019/03/08 15:02:57.325 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/03/08 15:02:57.328 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxTriggerManager registered.
2019/03/08 15:02:57.331 +0800 INFO [AzkabanWebServer] Bean azkaban.jmx.JmxExecutorManager registered.
2019/03/08 15:02:57.349 +0800 INFO [AzkabanWebServer] Bean org.apache.log4j.jmx.HierarchyDynamicMBean registered.
2019/03/08 15:02:57.349 +0800 INFO [AzkabanWebServer] ************* loginLoggerObjName is null, make sure there is a logger with nam
e azkaban.webapp.servlet.LoginAbstractAzkabanServlet
2019/03/08 15:02:57.350 +0800 INFO [ExecutorManager] Cleaning old logs from execution_logs
2019/03/08 15:02:57.353 +0800 INFO [AzkabanWebServer] Setting up web resource dir /u01/app/azkaban-3.50.0/azkaban-web-server-0.1.0-S
NAPSHOT/web
2019/03/08 15:02:57.383 +0800 INFO [ExecutorManager] Cleaning old log files before 2018-12-14T15:02:57.350+08:00
2019/03/08 15:02:57.385 +0800 INFO [ExecutorManager] Cleaned up 0 log entries.
2019/03/08 15:02:57.385 +0800 INFO [ExecutorManager] log clean up time: 0 seconds.
2019/03/08 15:02:57.435 +0800 INFO [JdbcTriggerImpl] Loading all triggers from db.
2019/03/08 15:02:57.437 +0800 INFO [JdbcTriggerImpl] Loaded 0 triggers.
2019/03/08 15:02:57.438 +0800 INFO [log] jetty-6.1.26
2019/03/08 15:02:58.736 +0800 INFO [ProjectManagerServlet] downloadBufferSize: 8192
2019/03/08 15:02:58.879 +0800 INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.UserManagerResource'
2019/03/08 15:02:58.881 +0800 INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.ProjectManagerResource'
2019/03/08 15:02:59.057 +0800 INFO [log] Started [email protected]:8443
2019/03/08 15:02:59.057 +0800 INFO [AzkabanWebServer] Server started
查看JPS，会出现AzkabanWebServer：
jps
6.3 登录管理界面
打开浏览器，输入https://10.200.4.117:8443，登录admin用户：
完毕。
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
智能推荐
【人脸检测 FPS 1000+】ubuntu下libfacedetection tensorrt部署_人脸检测 tensorrt-程序员宅基地
文章浏览阅读580次，点赞25次，收藏24次。【FPS 1000+】ubuntu下libfacedetection人脸检测 tensorrt加速部署_人脸检测 tensorrt
官方STM32F107_ETH_LwIP_V1.0.0开发包移植笔记_stm32f107 lwip 直连-程序员宅基地
文章浏览阅读1.2k次。一、LwIP移植记录lwip-1.4.1\src\core\ipv4\icmp.c 注释掉：#if CHECKSUM_GEN_ICMP新增加：#ifdef CHECKSUM_BY_HARDWARE代码如下：//#if CHECKSUM_GEN_ICMP// /* adjust the checksum */// if (iecho->chksum..._stm32f107 lwip 直连
Java开发笔记（十四）几种运算符的优先级顺序_大于小于和等于的优先级-程序员宅基地
文章浏览阅读7k次，点赞2次，收藏7次。到目前为止，我们已经学习了Java语言的好几种运算符，包括算术运算符、赋值运算符、逻辑运算符、关系运算符等基础运算符，并且在书写赋值语句时都没添加圆括号，显然是默认了先完成算术、逻辑、关系等运算，最后才进行赋值操作。也就是说，在这四类运算符当中，赋值运算符的优先级最低，那么其他三种运算符的优先级顺序又是如何排列的呢？首先来看算术运算符，Java中的算术运算符主要包括正号负号、加减乘除，以及取余数..._大于小于和等于的优先级
C++第7次作业_定义一个有10个的数组a,将数组a中的所有素数放在数组b,再对数组b排序。-程序员宅基地
文章浏览阅读407次。一：特殊三位数请输出满足条件n=a!+b!+c!的所有三位数n，其中，a、b、c分别是n的百、十、个位数。要求用自定义函数实现求阶乘。#includeusing namespace std;int f(int n){ int s=1,i; for(i=1;i<=n;i++) s=s*i; return s;} int main() {_定义一个有10个的数组a,将数组a中的所有素数放在数组b,再对数组b排序。
Matlab数值微分与数值积分，阿里巴巴Java面试都问些什么-程序员宅基地
文章浏览阅读900次，点赞19次，收藏16次。JVM，JAVA集合，JAVA多线程并发，JAVA基础，Spring原理，微服务，Netty与RPC，网络，日志，Zookeeper，Kafka，RabbitMQ，Hbase，MongoDB，Cassandra，设计模式，负载均衡，数据库，一致性哈希，JAVA算法，数据结构，加密算法，分布式缓存，Hadoop，Spark，Storm，YARN，机器学习，云计算…MongoDB，Cassandra，设计模式，负载均衡，数据库，一致性哈希，JAVA算法，数据结构，加密算法，分布式缓存。
python中字母数字的大小比较_python列表怎么比较大小-程序员宅基地
文章浏览阅读8.1k次。Python中可以使用cmp()函数比较两个列表的大小。cmp()函数语法：cmp(list1, list2)参数：list1 -- 比较的列表。list2 -- 比较的列表。返回值：如果比较的元素是同类型的,则比较其值,返回结果。如果两个元素不是同一种类型,则检查它们是否是数字。如果是数字,执行必要的数字强制类型转换,然后比较。如果有一方的元素是数字,则另一方的元素"大"(数字是"最小的")否则..._python中数字和字母比较大小
随便推点
谈谈Java集合ArrayList扩容，Java面试题集2024版-程序员宅基地
文章浏览阅读845次，点赞27次，收藏8次。***构造包含指定collection元素的列表，这些元素利用该集合的迭代器按顺序返回*如果指定的集合为null，throws NullPointerException。*/= 0) {} else {下面在我们分析 ArrayList 扩容时会降到这一点内容！二 一步一步分析 ArrayList 扩容机制这里以无参构造函数创建的 ArrayList 为例分析。
Mysql数据库之全文索引 FULLTEXT_mysql fulltext索引-程序员宅基地
文章浏览阅读392次，点赞6次，收藏6次。1、话说mysql innodb在5.6.4以后就有了fulltext全文索引。_mysql fulltext索引
JAVA四舍五入方法_string.format 四舍五入-程序员宅基地
文章浏览阅读1.5k次。JAVA中四舍五入方法_string.format 四舍五入
struts2是什么，有什么作用，怎么使用，有什么好处？_1、什么是struts2框架,它的作用是什么?-程序员宅基地
文章浏览阅读609次。注：参考文章：https://blog.csdn.net/qq_36754544/article/details/55062391struts2是一种重量级的框架，位于MVC架构中的controller，可以分析出来，它是用于接受页面信息然后通过内部处理，将结果返回。同时struts2也是一个web层的MVC框架，那么什么是struts2中的MVC呢？我们可以联想一下struts2的使用流程，我们需要先通过过滤器，调用URL判断使用哪个Action，再由Action决定返回结果，继而 提交..._1、什么是struts2框架,它的作用是什么?
【Matplotlib设置】Python绘图全局字体改为 Times New Roman_python画图调整字体为times-程序员宅基地
文章浏览阅读1w次，点赞16次，收藏14次。import matplotlib.pyplot as pltplt.rc('font',family='Times New Roman') 注意一定要在导入包之后就设置这个代码，否则可能不显示。参考文献[1]matplotlib 全局字体改为 Times New Roman_python画图调整字体为times
Aocoda-RCF7/F7 MINI飞控无法解锁的疑难杂症-使用 Betaflight 10.8.0调参软件地面站刷写固件以及AOCODAF722MINI 配置文件_betaflight configurator 10.9.0-程序员宅基地
文章浏览阅读2.7k次。今天调试了半天的Betaflight 10.8.0，在借鉴b站等平台大佬的经验基础上，艰难实现了Aocoda-RCF7/F7 MINI飞控解锁，没想到刚成功又出现了无法检测到陀螺仪的情况，真是一波不平一波又起，刚巧有发现了新版本的问世，也就是在昨天刚刚发布，顿时来了兴致，地址如下。但是其下载着实费劲。期待有更稳定更便捷的表现。下面还是回到我们的话题，就是这个f7的解锁问题。_betaflight configurator 10.9.0