Use the subprocess summarizes were studied how to operate the terminal from python.

The subprocess

This package for operating the terminal from python. Fact that the terminal can be operated is not things like do a shell script can be one way. It is dangerous and convenient too.

Since have been developed for many years call, check_call, but there seems to be various modules such as check_output, when you normally run using the subprocess.run (), recommended to use the subprocess.Popen () when running asynchronously It seems to be. Since Popen is a module for parallel processing, it has become to be able or to kill or waiting for the end of the child process.

The official document
https://docs.python.org/ja/3.6/library/subprocess.html#using-the-subprocess-module

This time do not do only immediately end processing use the subprocess.run ().

Trial environment

Windows 10

python 3.6

Ubuntu 18.04

python 3.6

To run the terminal command

ls
 

dir
 

The above command such as you can do the following in subprocess.run ().

import subprocess
 subprocess.run('ls')
 

import subprocess
 subprocess.run('dir', shell=True)
 

If the terminal on the display comes out command, you receive as a return value of the character string display. Since the character column can also things like taking out only a necessary part in the normal distribution.

import subprocess
 result = subprocess.run('ls')
 print(result)
 

import subprocess
 result = subprocess.run('dir', shell=True)
 print(result)
 

To run giving arguments to the terminal command

If you want to give the argument, but it was space-separated in the terminal will pass in the list in the subprocess.

result = subprocess.run(['ls', '-l'])
 print(result)
 

If there is more than one argument only list is getting longer.

result = subprocess.run(['ls', '-l', '-a'])
 print(result)
 

By the way, it can also be performed in only one. When it or not or there is an argument is easy likely to do better here of how to write.

result = subprocess.run(['ls'])
 print(result)
 

To run the shell script (batch file)

Since the terminal command anything can be executed, it will pass or arguments can also perform naturally shell script. The following have been carried out in two arguments.

result = subprocess.run(['hoge.sh', 'value1', 'value2'])
 print(result)
 

result = subprocess.run(['hoge.bat', 'value1', 'value2'])
 print(result)
 

To run python code

The fact that terminal command can be executed, you can also run python if python installation. The following is running, passing two arguments to hoge.py.

subprocess.run(['python', 'hoge.py', 'value1', 'value2'])
 

To run python code by specifying the conda environment

Since the python interpreter executable file, specify the PATH the environment that are running now if you run python You can also use the python of another conda environment. For example, E in a Windows environment:/Anaconda3 tf of conda environment that was installed in the/is E: since the/Anaconda3/envs/tf/python.exe, you can perform the following writing.

subprocess.run(['E:/Anaconda3/envs/tf/python.exe', 'hoge.py'])
 

In addition, to examine whether the python interpreter where you are is easy if where/which sure you conda activate in the terminal.

conda activate tf

where tf
 

conda activate tf

which tf
 

To check the gpu memory remaining

GPU usage to parse the string received if nvidia-smi can use environment were also some people that are doing the other I thought Dekirukana acquisition, and.
https://qiita.com/tomotaka_ito/items/1da001c98b46ecf28ec7

import subprocess
 import json

DEFAULT_ATTRIBUTES = (

    'index',

    'uuid',

    'name',

    'timestamp',

    'memory.total',

    'memory.free',

    'memory.used',

    'utilization.gpu',

    'utilization.memory'
 )

def get_gpu_info(nvidia_smi_path='nvidia-smi', keys=DEFAULT_ATTRIBUTES, no_units=True):

    nu_opt = '' if not no_units else ',nounits'

    cmd = '%s --query-gpu=%s --format=csv,noheader%s' % (nvidia_smi_path, ','.join(keys), nu_opt)

    output = subprocess.check_output(cmd, shell=True)

    lines = output.decode().split('\n')

    lines = [ line.strip() for line in lines if line.strip() != '' ]

    return [ { k: v for k, v in zip(keys, line.split(', ')) } for line in lines ]

if __name__ == "__main__":

    import pprint

    pprint.pprint(get_gpu_info())
 

We have look like a shell arts.

Open a new terminal to run the command

It's Linux in the gnome-terminal command, you can open a new terminal or command prompt and use the start command that it is Windows.

gnome-terminal ls -l
 

start dir/w
 

The above command can be run from a python so can be written as follows.

subprocess.run(['gnome-terminal', 'ls', '-l'])
 

subprocess.run(['start', 'dir', '/w'], shell=True)
 

People here if a long period of time-consuming process might be easy to monitor.

shell = True About

When you shell = True to run using the shell. Shell of the embedded command (Windows of dir Toka Toka start) will use in the shell = True because not be called only from the shell. But it means that she can do anything is the fact that to run in the shell will become a cause of vulnerability. It is not necessary to run a shell in the case, such as to run the file Leave the shell = False. The default is also OK if the shell = False So omitted.

For details, click here
https://docs.python.org/ja/3.6/library/subprocess.html#security-considerations

Also, if you shell = True will change the way given the argument. In the case of Linux, but gives the shell = False In the argument in the list, is set to give the argument that it is shell = True of space separated by a string. So caution is needed as you or I to unintended operation inadvertently got ignores the second and subsequent If you give in the list. For some reason you can not notice because the error is also not me out. In the case of Windows, shell = I will do the same for so space-delimited can be used and to True, we are able to receive normally the list even shell = True. Because to come also different stdio of behavior requires a variety of attention.

There is described in official documents
https://docs.python.org/ja/3.6/library/subprocess.html#popen-constructor

Conclusion

• subprocess Seriously convenient
• Beware the vulnerability
• Let's try

Modify History

20190903: shell = True has been fixed had been or not attached to the place where or attached to place unnecessary. shell = True we added the description of the.