on
Pentaho kitchen
Pentaho kitchen
정의
원격으로 job 을 실행시키는 모듈이다.
설치
data-intergration( carte ) 설치시 bundle 로 들어가 있다.
설정
pan 과 동일
general error
random 적으로 발생(같은 명령어 반복시 발생)
vfs 안쓰는데.. 어떻게 끄지? 객체를 vfs 단위로 취급하는듯
pentaho 9.1 cetified jdk version 은 jdk 8 (현재 환경과 같음)
미해결 https://jira.pentaho.com/browse/PDI-17149?page=com.atlassian.jira.plugin.system.issuetabpanels%3Aworklog-tabpanel 이미 등록된 버그 이지만 원인을 찾지 못하고 더 많은 cpu 와 ram 이 설치된 머신에서 문제가 없다고만 리포트됨
[bos@tlog-etl carte]$ kitchen.sh -listrep 15:36:38,237 INFO [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled 15:36:38,330 INFO [KarafInstance] ******************************************************************************* *** Karaf Instance Number: 2 at /home/bos/packages/data-integration/./syste *** *** m/karaf/caches/kitchen/data-1 *** *** Karaf Port:8803 *** *** OSGI Service Port:9052 *** ******************************************************************************* Nov 30, 2020 3:36:38 PM org.apache.karaf.main.Main launch INFO: Installing and starting initial bundles Nov 30, 2020 3:36:38 PM org.apache.karaf.main.Main launch INFO: All initial bundles installed and set to start Nov 30, 2020 3:36:39 PM org.apache.karaf.main.Main$KarafLockCallback lockAcquired INFO: Lock acquired. Setting startlevel to 100 2020/11/30 15:36:39 - Kitchen - Start of run. 2020/11/30 15:36:39 - RepositoriesMeta - Reading repositories XML file: /home/bos/packages/data-integration/repositories.xml List of repositories: #1 : postgres [Database repository] id=KettleDatabaseRepository #2 : file_repository [Pentaho repository | http://localhost:8080/pentaho] id=PentahoEnterpriseRepository 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pentaho-googledrive-vfs/pentaho-googledrive-vfs-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/elasticsearch-bulk-insert-plugin/elasticsearch-bulk-insert-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pdi-xml-plugin/pdi-xml-plugin-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/kettle-json-plugin/kettle-json-plugin-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/kettle-sap-plugin/kettle-sap-plugin-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pentaho-big-data-plugin/pentaho-big-data-legacy-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pentaho-big-data-plugin/pentaho-big-data-legacy-amazon-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pentaho-cassandra-plugin/pentaho-cassandra-plugin-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pdi-teradata-tpt-plugin/pdi-teradata-tpt-plugin-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/platform-utils-plugin/pdi-platform-utils-plugin-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/VerticaBulkLoader/vertica-bulkloader-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - General - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : Error while finding annotations for jar plugin: 'file:///home/bos/packages/data-integration/plugins/pdi-pur-plugin/pdi-pur-plugin-core-9.1.0.0-324.jar' 2020/11/30 15:36:39 - Plugin type - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to list jar files in plugin folder '/home/bos/.kettle/plugins' 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to get VFS File object for filename '/home/bos/.kettle/plugins' : Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to get VFS File object for filename '/home/bos/.kettle/plugins' : Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - ERROR (version 9.1.0.0-324, build 9.1.0.0-324 from 2020-09-07 05.09.05 by buildguy) : org.pentaho.di.core.exception.KettleFileException: 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to list jar files in plugin folder '/home/bos/.kettle/plugins' 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to get VFS File object for filename '/home/bos/.kettle/plugins' : Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to get VFS File object for filename '/home/bos/.kettle/plugins' : Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.PluginFolder.findJarFiles(PluginFolder.java:143) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.PluginFolder.findJarFiles(PluginFolder.java:117) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.JarFileCache.getFileObjects(JarFileCache.java:66) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.BasePluginType.findAnnotatedClassFiles(BasePluginType.java:394) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.BasePluginType.registerPluginJars(BasePluginType.java:702) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.BasePluginType.searchPlugins(BasePluginType.java:184) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.PluginRegistry.registerType(PluginRegistry.java:605) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:560) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:545) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:146) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:101) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:82) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.kitchen.Kitchen$1$1.call(Kitchen.java:95) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.kitchen.Kitchen$1$1.call(Kitchen.java:89) 2020/11/30 15:36:39 - Plugin type - at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2020/11/30 15:36:39 - Plugin type - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2020/11/30 15:36:39 - Plugin type - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 2020/11/30 15:36:39 - Plugin type - at java.lang.Thread.run(Thread.java:748) 2020/11/30 15:36:39 - Plugin type - Caused by: org.pentaho.di.core.exception.KettleFileException: 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Unable to get VFS File object for filename '/home/bos/.kettle/plugins' : Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.vfs.KettleVFS.getFileObject(KettleVFS.java:180) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.vfs.KettleVFS.getFileObject(KettleVFS.java:121) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.vfs.KettleVFS.getFileObject(KettleVFS.java:117) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.plugins.PluginFolder.findJarFiles(PluginFolder.java:125) 2020/11/30 15:36:39 - Plugin type - ... 17 more 2020/11/30 15:36:39 - Plugin type - Caused by: org.apache.commons.vfs2.FileSystemException: Could not find file with URI "/home/bos/.kettle/plugins" because it is a relative path, and no base URI was provided. 2020/11/30 15:36:39 - Plugin type - at org.apache.commons.vfs2.FileSystemException.requireNonNull(FileSystemException.java:87) 2020/11/30 15:36:39 - Plugin type - at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:733) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.vfs.ConcurrentFileSystemManager.resolveFile(ConcurrentFileSystemManager.java:91) 2020/11/30 15:36:39 - Plugin type - at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:682) 2020/11/30 15:36:39 - Plugin type - at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:637) 2020/11/30 15:36:39 - Plugin type - at org.pentaho.di.core.vfs.KettleVFS.getFileObject(KettleVFS.java:176) 2020/11/30 15:36:39 - Plugin type - ... 20 more ########################### ########################### ########################### 발생하지 않는 경우 ########################### ########################### [bos@tlog-etl carte]$ kitchen.sh -listrep 15:56:42,583 INFO [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled 15:56:42,681 INFO [KarafInstance] ******************************************************************************* *** Karaf Instance Number: 2 at /home/bos/packages/data-integration/./syste *** *** m/karaf/caches/kitchen/data-1 *** *** Karaf Port:8803 *** *** OSGI Service Port:9052 *** ******************************************************************************* Nov 30, 2020 3:56:43 PM org.apache.karaf.main.Main$KarafLockCallback lockAcquired INFO: Lock acquired. Setting startlevel to 100 2020/11/30 15:56:43 - Kitchen - Start of run. 2020/11/30 15:56:43 - RepositoriesMeta - Reading repositories XML file: /home/bos/packages/data-integration/repositories.xml List of repositories: #1 : postgres [Database repository] id=KettleDatabaseRepository #2 : file_repository [Pentaho repository | http://localhost:8080/pentaho] id=PentahoEnterpriseRepository
실행
repository list
[bos@tlog-etl carte]$ kitchen.sh -listrep 15:56:42,583 INFO [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled 15:56:42,681 INFO [KarafInstance] ******************************************************************************* *** Karaf Instance Number: 2 at /home/bos/packages/data-integration/./syste *** *** m/karaf/caches/kitchen/data-1 *** *** Karaf Port:8803 *** *** OSGI Service Port:9052 *** ******************************************************************************* Nov 30, 2020 3:56:43 PM org.apache.karaf.main.Main$KarafLockCallback lockAcquired INFO: Lock acquired. Setting startlevel to 100 2020/11/30 15:56:43 - Kitchen - Start of run. 2020/11/30 15:56:43 - RepositoriesMeta - Reading repositories XML file: /home/bos/packages/data-integration/repositories.xml List of repositories: #1 : postgres [Database repository] id=KettleDatabaseRepository #2 : file_repository [Pentaho repository | http://localhost:8080/pentaho] id=PentahoEnterpriseRepository
job list
[bos@tlog-etl carte]$ kitchen.sh -rep=postgres -user=admin -pass=admin -listjobs 16:03:02,185 INFO [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled 16:03:02,296 INFO [KarafInstance] ******************************************************************************* *** Karaf Instance Number: 2 at /home/bos/packages/data-integration/./syste *** *** m/karaf/caches/kitchen/data-1 *** *** Karaf Port:8803 *** *** OSGI Service Port:9052 *** ******************************************************************************* Nov 30, 2020 4:03:03 PM org.apache.karaf.main.Main$KarafLockCallback lockAcquired INFO: Lock acquired. Setting startlevel to 100 2020/11/30 16:03:03 - Kitchen - Start of run. 2020/11/30 16:03:03 - RepositoriesMeta - Reading repositories XML file: /home/bos/packages/data-integration/repositories.xml Job -rep=postgres -user=admin -pass=admin -listjob
logfile
kitchen.sh -rep=postgres -user=admin -pass=admin -listjobs -logfile=jobs.out
from http://choelhee.tistory.com/57 by ccl(A) rewrite - 2021-10-01 16:27:06