증상
Airflow webserver, scheduler가 작동하지 않음
Error log
Traceback (most recent call last):
File "/home/keti/.local/lib/python3.6/site-packages/lockfile/pidlockfile.py", line 77, in acquire
write_pid_to_pidfile(self.path)
File "/home/keti/.local/lib/python3.6/site-packages/lockfile/pidlockfile.py", line 161, in write_pid_to_pidfile
pidfile_fd = os.open(pidfile_path, open_flags, open_mode)
FileExistsError: [Errno 17] File exists: '/home/keti/airflow/airflow-webserver-monitor.pid'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/keti/anaconda3/envs/airflow/bin/airflow", line 8, in <module>
sys.exit(main())
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/airflow/__main__.py", line 40, in main
args.func(args)
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/airflow/utils/cli.py", line 91, in wrapper
return f(*args, **kwargs)
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/airflow/cli/commands/webserver_command.py", line 467, in webserver
with ctx:
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/daemon/daemon.py", line 389, in __enter__
self.open()
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/daemon/daemon.py", line 381, in open
self.pidfile.__enter__()
File "/home/keti/.local/lib/python3.6/site-packages/lockfile/__init__.py", line 197, in __enter__
self.acquire()
File "/home/keti/anaconda3/envs/airflow/lib/python3.6/site-packages/daemon/pidfile.py", line 57, in acquire
super().acquire(timeout, *args, **kwargs)
File "/home/keti/.local/lib/python3.6/site-packages/lockfile/pidlockfile.py", line 88, in acquire
self.path)
lockfile.AlreadyLocked: /home/keti/airflow/airflow-webserver-monitor.pid is already locked
Solution
.pid 파일을 지워주기만 하면된다.
$ sudo rm airflow-scheduler.pid airflow-webserver-monitor.pid
'Data Engineering > Airflow' 카테고리의 다른 글
Airflow의 parallelism과 dag_concurrency (0) | 2022.05.26 |
---|---|
Dockeroperator의 Bind mount을 활용한 Airflow 운영 (0) | 2021.09.05 |
Airflow tutorial - 1 (0) | 2021.07.10 |