Run subprocess asynchronously

默认情况下 subprocess.run() 是 sync 模式. 也就是一条命令执行完毕后才会执行下一条命令. 但是有时候我们需要异步执行多条命令, 这时候我们可以使用 subprocess.Popen() 来实现.

例如在下面的例子中我们有两个脚本, 分别是 sleep 5 秒和 10 秒.

script1.py

import time

print("start script 1")
time.sleep(5)
print("end script 1")

script2.py

import time

print("start script 2")
time.sleep(10)
print("end script 2")

我们可以用下面的脚本来异步执行这两个脚本. 总共花费了 10 秒钟.

subprocess_run_async_mode_example.py

# -*- coding: utf-8 -*-

import subprocess
from datetime import datetime

def run_script_async(script):
    process = subprocess.Popen(['python', script])
    return process

# Start the timer
start_time = datetime.now()

# Run script1 and script2 concurrently
script1_process = run_script_async("script3.py")
script2_process = run_script_async("script4.py")

# Wait for both scripts to complete
response1 = script1_process.wait()
response2 = script2_process.wait()
print(f"{response1 = }")
print(f"{response2 = }")

# Calculate the total execution time
execution_time = (datetime.now() - start_time).total_seconds()

print(f"Total execution time: {execution_time:.2f} seconds")

还是用 subprocess_run_async_mode_example.py 脚本, 但是我们换成下面两个 script, 其中一个会运行到一半抛出异常. 你可以看到由于是异步执行, 所以另一个脚本还是会继续执行.

script3.py

import time

print("start script 3")
time.sleep(5)
raise Exception("script 3 raises an error!")
print("end script 3")

script4.py

import time

print("start script 4")
time.sleep(10)
print("end script 4")