Airflow PYTHONPATH

Built-in PYTHONPATH entries in Airflow

airflow 다음과 같이 3개 경로를 PYTHONPATH로 설정함

Airflow, when running dynamically adds three directories to the sys.path:

• The dags folder: It is configured with option dags_folder in section [core].
• The config folder: It is configured by setting AIRFLOW_HOME variable ({AIRFLOW_HOME}/config) by default.
• The plugins Folder: It is configured with option plugins_folder in section [core].

​

Inspecting your PYTHONPATH loading configuration

airflow info 명령어 수행 결과의 python_path에서 확인할 수 있음

비고)airflow를 수행하기 전 export PYTHONPATH=/path를 수행한 경우 airflow info 명령어로 확인할 수 없음

​

When are plugins (re)loaded?

https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/plugins.html#plugins-loading

Plugins — Airflow Documentation

 

Adding directories to the PYTHONPATH

환경 변수 PYTHONPATH를 사용하여 sys.path에 추가할 추가 디렉터리를 지정할 수 있습니다.

다음 명령을 사용하여 프로젝트 루트에 대한 경로를 제공하여 Python 셸을 시작합니다.

PYTHONPATH=/home/arch/projects/airflow_operators python

>>> import sys
>>> from pprint import pprint
>>> pprint(sys.path)
['',
 '/home/arch/projects/airflow_operators'
 '/home/arch/.pyenv/versions/3.8.4/lib/python37.zip',
 '/home/arch/.pyenv/versions/3.8.4/lib/python3.8',
 '/home/arch/.pyenv/versions/3.8.4/lib/python3.8/lib-dynload',
 '/home/arch/venvs/airflow/lib/python3.8/site-packages']

>>> import airflow_operators
Hello from airflow_operators

PYTHONPATH=/home/arch/projects/airflow_operators airflow info

 

https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/modules_management.html#adding-directories-to-pythonpath