人工智能 Python调试之IPython工具

hooooooooha · April 18, 2020 · 9 hits

Python工程师需要快速验证代码的运行结果是否符合预期，最为方便的做法就是使用Python自带的交互式解释器，但是因为其有很多弊端，所以IPython就是不二的选择了。

IPython 的官方文档地址

使用 IPython 有哪些好处

Python调试之IPython工具

自带 Python 解释器的弊端
- 不能在退出时保存历史记录以备未来查询
- 不支持Tab自动补全，其实是可以支持的
- 不方便在交互式环境下执行Shell命令
- 不能快速获得模块、函数、类、文档、原始代码信息


[[email protected] ~]$ python
Python 3.6.4 (default, Mar 28 2018, 22:01:18)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import rlcompleter, readline
>>> readline.parse_and_bind('tab: complete')

IPython 提供的常用功能
- 强大的交互式shell命令
- 基于浏览器的notebook支持代码、文字、数学表达式、内嵌图和其他丰富的媒体
- 支持交互式数据可视化的和GUI工具包的使用
- 灵活的、可嵌入的解释器方便加入到自己的项目
- 支持并行计算的简单易用、高性能的工具
常用 Magic 函数

常用 Magic 函数	解释说明
`%hist`	查看历史记录
`%save`	把某些历史记录保存到文件中
`%pwd`	显示当前位置
`%timeit`	获得程序执行时间
`%rehashx`	把`$PATH`中的可执行命令都更新进别名系统，这样就可以不加感叹号调用
`%debug`	激活交互的调试器
`%edit`	使用编辑器打开，但需要设定`EDITOR`这个环境变量
`%run`	执行引入的模块
`%load`	把外部代码加载进来
`%macro`	把历史记录、文件等封装为宏，以便未来重新执行
`%alias`	设置命令别名
`%bookmark`	设置书签
`%store`	存储设置的命令别名
`%logstart/%logoff`	记录会话或日志记录，退出后还可以回到之前的状态

IPython 提供的 magic 函数
- 新版本的IPython可以不添加%符号，因为系统自己设置automagic为ON，表示系统自动添加magic符号。
- 这里我们只重点介绍line magics，而cell magics请自行查看官方文档说明。

In [1]: %lsmagic
Out[1]:
# line magics: %x
# 主要涉及一些功能的命令
%alias  %alias_magic  %autocall  %autoindent  %automagic  %bookmark  %cat  %cd
%clear  %colors  %config  %cp  %cpaste  %debug  %dhist  %dirs  %doctest_mode
%ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk
%ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop
%ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv
%notebook  %page  %paste  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2
%popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd
%pycat  %pylab  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset
%reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system
%tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

# cell magics: %%xx
# 主要是渲染ipython notebook页面效果以及执行某语言的代码
%%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js
%%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3
%%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

In [5]: automagic
Automagic is OFF, % prefix IS needed for line magics.

In [8]: %automagic
Automagic is ON, % prefix IS NOT needed for line magics.

# line magic
In [1]: %time 2**128
CPU times: user 2 µs, sys: 1 µs, total: 3 µs
Wall time: 5.01 µs
Out[1]: 340282366920938463463374607431768211456L

# cell magic
In [2]: %%ruby
   ...: p "hello"
      ...:
      "hello"

# line and cell magic
In [5]: %%time
   ...: 2**128
   ...:
   CPU times: user 4 µs, sys: 0 ns, total: 4 µs
   Wall time: 9.06 µs
   Out[5]: 340282366920938463463374607431768211456L

1. 配置解释器

修改配置文件过程

# 在用户家目录下的默认没有IPython的配置文件
[[email protected] ~]$ ll /Users/escape/.ipython/profile_default/
total 256
drwxr-xr-x  5 escape  staff    160  6 27 11:22 db/
-rw-r--r--  1 escape  staff  77824  6 27 11:47 history.sqlite
drwxr-xr-x  2 escape  staff     64  3 24 19:57 log/
drwx------  2 escape  staff     64  3 24 19:57 pid/
drwx------  2 escape  staff     64  3 24 19:57 security/
drwxr-xr-x  3 escape  staff     96  3 24 19:57 startup/

# 创建IPython的配置文件
[[email protected] ~]$ ipython profile create
[ProfileCreate] Generating default config file: '/Users/escape/.ipython/profile_default/ipython_config.py'

# 配置IPython参数
[[email protected] ~]$ vim ~/.ipython/profile_default/ipython_config.py
# 用于自动重载配置的宏、变量、别名等，无需使用store -r命令
c.StoreMagics.autorestore = True
# 用于自动重载文件，不用我们退出IPython再进入了
c.InteractiveShellApp.exec_lines = ['%autoreload 2']
c.InteractiveShellApp.extensions = ['autoreload']
......

storemagic
- 可以持久化宏、变量和别名
autoreload
- 可以让我们不退出 IPython 就动态修改代码

# storemagic
# 保存之后重新打开依旧可以使用
In [1]: l = [1, 2, 3]
In [2]: store l
Stored 'l' (list)
In [3]: exit()

In [1]: l
Out[1]: [1, 2, 3]

# autoreload
# 重新编辑对于的文件，修改输出
In [1]: from test import run

In [2]: run()
Out[2]: 1

In [3]: %edit test.py
Editing... done. Executing edited code...

In [4]: run()
Out[4]: 2

2. Tab 自动补全

IPython可以自动检查对象的属性，通过object_name.<TAB>列出全部的子属性，再次使用Tab切换到对于的属性上，输入输出即可选中。

python-ipyton

python-ipyton

3. 获取对象信息

在想要查看的对象后面加上一个问号，只显示对象的签名、文档字符串及代码文件的位置信息
在想要查看的对象后面加上两个问号，可以直接显示源代码信息，这样节省时间搜索对于代码

In [1]: import os.path

# 对象后面加上一个问号
In [2]: os.path?
Type:        module
String form: <module 'posixpath' from '/Users/wenpan/.pyenv/versions/3.6.4/lib/python3.6/posixpath.py'>
File:        ~/.pyenv/versions/3.6.4/lib/python3.6/posixpath.py
Docstring:
Common operations on Posix pathnames.

Instead of importing this module directly, import os and refer to
this module as os.path.  The "os.path" name is an alias for this
module on Posix systems; on other systems (e.g. Mac, Windows),
os.path provides the same operations in a manner specific to that
platform, and is an alias to another module (e.g. macpath, ntpath).

Some of this can actually be useful on non-Posix systems too, e.g.
for manipulation of the pathname component of URLs.

# 对象后面加上两个问号
In [3]: os.path??
Type:        module
String form: <module 'posixpath' from '/Users/escape/.pyenv/versions/3.6.4/lib/python3.6/posixpath.py'>
File:        ~/.pyenv/versions/3.6.4/lib/python3.6/posixpath.py
Source:
"""Common operations on Posix pathnames.

Instead of importing this module directly, import os and refer to
this module as os.path.  The "os.path" name is an alias for this
module on Posix systems; on other systems (e.g. Mac, Windows),
os.path provides the same operations in a manner specific to that
platform, and is an alias to another module (e.g. macpath, ntpath).

Some of this can actually be useful on non-Posix systems too, e.g.
for manipulation of the pathname component of URLs.
"""

import os
import sys
import stat
import genericpath
from genericpath import *

__all__ = ["normcase","isabs","join","splitdrive","split","splitext",
           "basename","dirname","commonprefix","getsize","getmtime",
           "getatime","getctime","islink","exists","lexists","isdir","isfile",
           "ismount", "expanduser","expandvars","normpath","abspath",
           "samefile","sameopenfile","samestat",
           "curdir","pardir","sep","pathsep","defpath","altsep","extsep",
           "devnull","realpath","supports_unicode_filenames","relpath",
           "commonpath"]
......

4. 调用系统命令

IPython在line magics中定义了一些常见的Shell命令可供使用，但是其他的命令则需要通过添加!使用
可以通过IPython自带的%rehashx函数，重载系统Shell命令，之后就无需在添加!符号了

In [1]: ls
__pycache__/             delete_server_log.py*    jumpserver_on_docker.sh* test.py                  update_chis.sh*

In [2]: ll
total 32
drwxr-xr-x  3 escape  staff    96  6 26 21:33 __pycache__/
-rwxr--r--  1 escape  staff     0  6 19 11:01 delete_server_log.py*
-rwxr--r--@ 1 escape  staff  6854  5 27 13:57 jumpserver_on_docker.sh*
-rw-r--r--  1 escape  staff    17  6 27 09:50 test.py
-rwxrwxrwx  1 escape  staff   128  5 25 15:40 update_chis.sh*

In [3]: !uptime
10:31  up 14 days, 21:25, 2 users, load averages: 2.70 2.88 2.97

In [4]: %rehashx

In [5]: uptime
10:31  up 14 days, 21:25, 2 users, load averages: 2.63 2.86 2.96

In [6]: date
2018年 6月28日 星期三 10时31分46秒 CST

5. 查看历史记录

IPython把输入的历史记录存储在个人设置目录下的history.sqlit文件中，并且可以结合%rerun、%macro、%save等magic函数一起使用。
尤为重要的是，它将最近的三次执行记录绑定在、和__这三个变量上了。搜索历史记录是，可以使用ctrl+r、ctrl+n、ctrl+p等快捷键操作。

In [1]: a = 1

In [2]: b = 2

# 打印出历史记录
In [3]: hist
1: a = 1
2: b = 2

# 掉历史记录中的序号
In [4]: hist -n
a = 1
b = 2

6. edit 编辑文件

edit魔术方法可以在命令行中直接编辑也可以编辑对于文件

In [1]: a = 1

In [2]: b = 2

In [3]: c = 3

In [4]: edit 2:3
IPython will make a temporary file named: /var/folders/5h/v75wkmc16_v0l9lj36_78qk80000gn/T/ipython_edit_kqrqbsp8/ipython_edit_6orujval.py
Editing... done. Executing edited code...
Out[4]: 'b = 4n'

In [5]: b
Out[5]: 4

In [6]: edit note.py
Editing... done.

In [7]: run note.py

In [8]: l
Out[8]: range(0, 12)

7. 设置命令别名

设置命令别名使用alias命令
设置的别名可以存储起来，不然重启IPython的时候就无法再使用了

In [1]: alias largest ls -1sSh | grep %s

In [2]: %largest t
total 32
 8 update_chis.sh
 8 test.py
 0 delete_server_log.py

# 设置的别名可以存储起来
In [5]: store largest
Alias stored: largest (ls -1sSh | grep %s)

# 可以设置书签
In [6]: bookmark root /Users/escape

In [7]: cd root
(bookmark:root) -> /Users/escape
/Users/escape

In [8]: pwd
Out[8]: '/Users/escape'

In [9]: save hist.py 3:5
The following commands were written to file `hist.py`:
sotre largest
sotre %larg

# 重新进入IPython的环境，重载之前保存的别名
In [1]: store -r

In [2]: largest t
total 32
 8 update_chis.sh
 8 test.py
 0 delete_server_log.py

8. 打开调试模式

IPython提供了一个Python debugger的接口。在自动pdb呼叫启用的情况下，当Python遇到一个未处理的异常时Python debugger就会自动启动。
IPython的作者说有时候当他需要在某行代码处debug时，他会在开始debug的地方放一个表达式1/0。启用pdb，在IPython中运行代码。当解释器处理到1/0那一行时，就会产生一个 ZeroDivisionError异常，然后他就在指定的代码处被带到一个debugging session中了。

In [1]: a = 1

In [2]: b = 0

In [3]: a / b
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-3-d8e10abd5ab6> in <module>()
----> 1 a / b

ZeroDivisionError: division by zero

In [4]: debug
> <ipython-input-3-d8e10abd5ab6>(1)<module>()
----> 1 a / b

ipdb> p a
1
ipdb> p b
0

9. 执行程序代码

执行程序代码可以使用如下方式：run <源文件> <运行源文件所需参数>

# [-n]参数
阻止运行源文件代码时__name__变量被设为”__main__”。
这会防止if __name__ == "__main__":块中的代码被执行

# [-i]参数
源文件就在当前IPython的名字空间下运行而不是在一个新的名字空间中。
如果你需要源代码可以使用在交互式session中定义的变量就会很有用。

# [-p]参数
使用Python的profiler模块运行并分析源代码。
使用该选项代码不会运行在当前名字空间。

In [9]: load hist.py

In [10]: # %load hist.py
    ...: sotre largest
    ...: sotre %largest
    ...:

In [11]: run hist.py

10. 用户会话保持

用户会话保持，也可以称为会话保持

# 打开日志记录或用户会话保持
In [1]: %logstate
Logging has not been activated.

In [2]: %logstart
Activating auto-logging. Current session state plus future input saved.
Filename       : ipython_log.py
Mode           : rotate
Output logging : False
Raw input log  : False
Timestamping   : False
State          : active

In [3]: a = 1

In [4]: b = 2

In [5]: c = 3

In [6]: %logoff
Switching logging OFF

# 重新打开IPython工具，之前定义的参数仍然可以使用
[[email protected] ~]$ ipython -i ipython_log.py
Python 3.6.4 (default, Mar 28 2018, 22:01:18)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.
Logging has not been activated.
Activating auto-logging. Current session state plus future input saved.
Filename       : ipython_log.py
Mode           : rotate
Output logging : False
Raw input log  : False
Timestamping   : False
State          : active
Switching logging OFF

In [1]: a
Out[1]: 1

In [2]: b
Out[2]: 2

In [3]: c
Out[3]: 3

11. 用户自定义宏

宏允许用户为一段代码定义为一个名字，这样你可以在以后使用这个名字来运行这段代码。就像在magic关键字edit中提到的，列表切片法也适用于宏定义。

In [1]: l = [1, 2]

In [2]: for i in l:
   ...:     print(i)
   ...:
1
2

In [5]: %macro print_l 1:3
Macro `print_l` created. To execute, type its name (without quotes).
=== Macro contents: ===
l = [1, 2]
for i in l:
    print(i)


In [6]: print_l
1
2

document.querySelectorAll('.github-emoji') .forEach(el => { if (! el.dataset.src) { return; } const img = document.createElement('img'); img.style = 'display:none ! important;'; img.src = el.dataset.src; img.addEventListener('error', () => { img.remove(); el.style.color = 'inherit'; el.style.backgroundImage = 'none'; el.style.background = 'none'; }); img.addEventListener('load', () => { img.remove(); }); document.body.appendChild(img); });

No Reply at the moment.

You need to Sign in before reply, if you don't have an account, please Sign up first.