Do It For You

comsumer콘솔 :

~/kafka/logs$ ~/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic pjm1 --from-beginning > pjm1.log

producer콘솔 :
echo '[  {    "name": "jinmyung",    "age": 500,    "secretIdentity": "Andrew",    "powers": [      "Radiation resistance",      "Turning tiny",      "Radiation blast"    ]  }]'  | ~/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic pjm1

https://streamsets.com/ 로그인

Deployments 메뉴를 이용하여 deployment task진행

: A deployment is a group of identical engine instances deployed within an environment.

intall script를 로컬환경에서 실행하면

streamSets collector를 다운로드 받음.

위 스크립트가 수행되면 아래 경로의 파일을 다운로드 함. (오류가 있으면 브라우저를 통해 다운로드 진행)

Starting download of 

https://archives.streamsets.com/datacollector/4.1.0/tarball/streamsets-datacollector-core-4.1.0.tgz

dcans@dcans:~/.streamsets/download/dc/collector$ sudo bin/streamsets dc

Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS
INFO - Starting engine
Logging initialized @5466ms to org.eclipse.jetty.util.log.Slf4jLog
Running on URI : 'http://dcans:18630'

-------------------------------------------------------

StreamSets collector를 구동하기 위해 jdk 1.8이상 필요.

버전이 맞지 않아 기존 jdk 삭제후 재설치 함

jdk삭제하는 법

Remove OpenJDK,

sudo apt remove openjdk*

Remove OpenJDK along with dependencies,

sudo apt remove --auto-remove openjdk*

Remove OpenJDK and the configuration files

sudo apt purge openjdk*