How To Use Python Subprocess Module Tutorial

When we run python program, we are creating and running a process to execute the program. Like Linux process, a python process can fork a child process and let the child process to execute another program. In Python, we fork a child process through the subprocess module in the standard library and run an external program in that child process. This article will tell you how to use the python subprocess module

1. Python Subprocess Module Method Introduction.

The subprocess module defines several functions for creating child processes. These functions create child processes in different ways, so we can select one of them as needed. In addition, subprocess also provides some tools for managing standard stream and pipe to use text communication between processes.

1.1 subprocess.run() Method.

The run method is the method which can execute an external program in a child process. It will return an instance of subprocess.CompletedProcess class. The syntax format of the run method is as follows.

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, text=None, env=None, universal_newlines=None)
  1. args : Represents the command to execute. Must be a string,  list or tuple.
  2. stdin, stdout, stderr : Standard input, output, and error for subprocesses. Its value can be subprocess.PIPE (means create a new pipe for the child process) , subprocess.DEVNULL ( means to use os.devnull ), an existing file descriptor, an open file object, or None (do nothing). In addition, stderr can be merged into stdout for output together.
  3. timeout : Set the command timeout time. If the command execution time is timeout, the child process will be killed and a subprocess.TimeoutExpired exception will be thrown.
  4. check : If this parameter is set to True and the process exit status code is not 0, a CalledProcessError exception will be thrown.
  5. encoding : If this parameter is specified, stdin, stdout, and stderr can receive string data which is encoded use this encoding method. Otherwise only Bytes type data will be received.
  6. shell : If this parameter is True, then the specified command is executed through the shell of the operating system.

Below is subprocess run method examples.

Invoke subprocess module’s run method to execute a shell command. And return the subprocess.CompletedProcess¬†object.

# first you should import the subprocess module.
>>> import subprocess

# invoke the run method.
>>> ret = subprocess.run(["dir","/p"],shell=True)
 Volume in drive C has no label.
 Volume Serial Number is 7689-56A2

 Directory of C:\Users\zhaosong

21/08/2020  11:33    <DIR>          .
21/08/2020  11:33    <DIR>          ..
......
               3 File(s)            160 bytes
              30 Dir(s)  204,775,772,160 bytes free
>>>
# print out the returned CompletedProcess object.
>>> ret
CompletedProcess(args=['dir', '/p'], returncode=0)
# print the returned code value.
>>> ret.returncode
0

In below example, it will write the exeternal program execution result text into a file.

>>> import subprocess
>>>
# open a file object to write external program's execution result.
>>> file = open("./test.txt","w")
>>>
# specify the file object to run method's stdout parameter.
>>> ret = subprocess.run(["dir","/p"],shell=True,stdout=file)
>>>
# print out the returned CompletedProcess object.
>>> ret
CompletedProcess(args=['dir', '/p'], returncode=0)

1.2 subprocess.Popen Class.

Popen class is the core of subprocess module, it is used to create and manage sub processes. Below is the Popen class’s constructor parameters.

class subprocess.Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0,restore_signals=True,start_new_session=False, pass_fds=(), *, encoding=None, errors=None)
  1. args : Shell command, which can be a string or sequence type (e.g. list, tuple)
  2. bufsize : Buffer size. Used when creating a pipe object for a standard stream, it’s value can be 0 (do not use buffer), 1 (¬†represents a line buffer, available only if universal_newlines=True ). Positive number : Indicate the buffer size, Negative number : Indicate use the system default buffer size.
  3. stdin, stdout, stderr : Represents the standard input, output, and error handlers for the program executed in the child process.
  4. preexec_fn : Valid on Unix platforms only, it specifies a callable object that will be called before the child process runs.
  5. shell : If this parameter is true, the specified command is executed through the shell of the operating system.
  6. cwd : Used to set the current working directory of the child process.
  7. env : Used to specify the child process’s environment variable. If you set env = None, then the child process will inherit the environment variables from the parent process.

The Popen class has below methods.

  1. poll() : Check whether the process is terminated. If the process is terminated, return the returncode, otherwise return None.
  2. wait(timeout) : Wait for the child process to terminate.
  3. communicate(input,timeout) : Interact with child process, send data to stdin and read data from stdout.
  4. send_signal(singnal) : Send signal to child process.
  5. terminate() : Stop the child process, that is, send the SIGTERM signal to the child process.
  6. kill() : Kill the child process. Send SIGKILL signal to child process.

Below are some examples.

# first import subprocess module.
>>> import subprocess
>>>
# create an instance of subprocess.Popen class.
>>> pObj = subprocess.Popen('dir /p', shell=True)
>>>  Volume in drive C has no label.
......

# check whether child process complete or not.
>>> pObj.poll()
0
# print out the return code.
>>> pObj.returncode
0
>>>

In below example, it will define a custom function and in this function it will create a child process to run the command.

import subprocess 

# define a function, the input parameter is the command string that will be executed.
def run_in_chile_process(command):

    # create subprocess.Popen instance to run the command in OS shell.
    pObj = subprocess.Popen(command,shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE,stderr=subprocess.PIPE,encoding="utf-8")
   
    # wait for the child process to complete, the time out time is 2 seconds.  
    pObj.wait(2)
    
    # if child process complete successfully.
    if pObj.poll() == 0:
    
        print("success")
        
        try:
            out, err = pObj.communicate()
        
        except subprocess.TimeoutExpired:
            
            pObj.kill()
            
            out, err = pObj.communicate()
            
        print("out = ", out)
        
        print("err = ", err)
            
    
    else:
    
        print("fail")



if __name__ == '__main__':
    run_in_chile_process("java -version")

Below is above example output.

success
out =  
err =  java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

Reference

  1. How To Fix Python Subprocess Run Error FileNotFoundError: [winerror 2] The System Cannot Find The File Specified

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.