Oozie Perfomance Tunning

Callable
- 비동기적으로 실행되는 작업 단위
- 작업: 워크플로우, 코디네이터, 시스템 내부 작업 

CallableQueueService
- Callable 객체들을 큐에 넣고 비동기적으로 실행하는 서비스
- 큐 크기와 스레드 수를 조정하여 성능 최적화 가능

주요설정
- oozie.service.CallableQueueService.queue.size: Callable 큐의 최대 크기 (기본값: 10000)
- oozie.service.CallableQueueService.threads: Callable 실행을 위한 스레드 수 (기본값: 10)
- oozie.service.CallableQueueService.callable.concurrency: 특정 Callable 유형의 최대 동시성 (기본값: 3)

성능 영향:
- 큐 크기와 스레드 수를 증가시키면 동시에 처리할 수 있는 작업 수가 증가
- 복잡한 워크플로우나 많은 코디네이터가 있는 경우 이 값들을 조정하여 성능 개선 가능

관련 설정:
- oozie.service.CallableQueueService.threads와 oozie.service.CallableQueueService.callable.concurrency는 함께 조정되어야 함

리소스 고려
- oozie.service.CallableQueueService.threads , oozie.service.CallableQueueService.callable.concurrency
- 이 값을 증가시킬 때는 서버의 리소스 용량을 고려해야함. 과도한 증가는 시스템 부하를 초래함

oozie.service.CallableQueueService.threads
- 용도
 : Oozie의 CallableQueueService에서 사용되는 스레드 수를 지정
 : 이 스레드는 Callable 객체들을 실행하는 데 사용
- 적용
 : 이 값을 증가시키면 Oozie가 동시에 처리할 수 있는 작업의 수가 증가
- 조정 필요성
 : 복잡한 워크플로우와 많은 코디네이터가 있는 워크로드의 경우, 이 값을 200까지 증가시키는 것이 권장


oozie.service.CallableQueueService.callable.concurrency
- 용도
: 특정 Callable 유형에 대한 최대 동시성을 제어
: 이 값을 증가시키면 Oozie가 동시에 처리할 수 있는 작업의 수가 증가
- 적용 범위:
 : 각 명령(submit, start, run, signal, job, jobs, suspend, resume 등)
 : 각. 액션 유형(Map-Reduce, Pig, SSH, FS, sub-workflow 등)
- 조정 필요성: 복잡한 워크플로우와 많은 코디네이터가 있는 워크로드의 경우, 이 값을 증가시키는 것이 권장됩니다.