【CPython3.6源码分析】Python 一般字节码执行

参考

前情提要

1
2
3
4
5
6
7
8
9
10
11
12
_PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
{

for (;;) {
// ceval.c.1267
switch (opcode) {
TARGET(LOAD_FAST) { ... }
TARGET(LOAD_CONST) { ... }
...
}
}
}

在前面我们提到,解释器会在 _PyEval_EvalFrameDefault进入for(;;)死循环,不断加载字节码指令,并执行。本章将通过几个常用的字节码指令,来了解 Python 字节码指令执行的逻辑。阅读本章前需了解PyCodeObject/PyFrameObject

1
2
3
4
5
6
7
8
9
10
11
12
# demo.py
a = 1

>>>co = compile(open('demo.py').read(),'demo.py','exec'); import dis; dis.dis(co)

1 0 LOAD_CONST 0 (1)
2 STORE_NAME 0 (a)

2 4 BUILD_MAP 0
6 STORE_NAME 1 (b)
8 LOAD_CONST 1 (None)
10 RETURN_VALUE

字节码序列:

  • 第一列:字节码对应源码中的行号
  • 第二列:当前字节码指令在 co_code 中的偏移位置
  • 第三列:当前字节码指令
  • 第四列:oparg,指令参数
  • 最后一列:当前字节码指令的参数实际内容

LOAD_CONST

1
2
3
4
5
6
7
8
9
10
11
12
/*
1 0 LOAD_CONST 0 (1)

opcode:: LOAD_CONST (consti)
Pushes ``co_consts[consti]`` onto the stack.
*/
TARGET(LOAD_CONST) {
PyObject *value = GETITEM(consts, oparg); // 加载序号为 oparg 的元素
Py_INCREF(value); // 增加引用
PUSH(value); // 入栈
FAST_DISPATCH(); // 继续循环
}

consts = f->f_code->co_consts,即 CodeObject 中的所有常量,一张常量表。LOAD_CONST 完成后,栈顶增加元素 1,栈顶指针下移。

STORE_NAME

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
/*
2 STORE_NAME 0 (a)

opcode:: STORE_NAME (namei)
Implements ``name = TOS``. *namei* is the index of *name* in the attribute
:attr:`co_names` of the code object.
*/
TARGET(STORE_NAME) {
PyObject *name = GETITEM(names, oparg); // names = f->co->co_names;
PyObject *v = POP(); // 从运行时栈中获取值
PyObject *ns = f->f_locals;
int err;
if (ns == NULL) {
PyErr_Format(PyExc_SystemError,
"no locals found when storing %R", name);
Py_DECREF(v);
goto error;
}
// 将 符号name-值v 的映射关系存储到 locals 中
if (PyDict_CheckExact(ns))
err = PyDict_SetItem(ns, name, v);
else
err = PyObject_SetItem(ns, name, v);
Py_DECREF(v);
if (err != 0)
goto error;
DISPATCH();
}

注意到,上一步刚进行了压栈操作,第二步就进行了出栈。然后从 co_names中获取第0个元素,作为字典的k-v,映射到 locals 中。此时,栈空,f_locals 指针依然指向开始位置。

BUILD_MAP/BUILD_LIST

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
opcode:: BUILD_MAP (count)
Pushes a new dictionary object onto the stack. Pops ``2 * count`` items
so that the dictionary holds *count* entries:
``{..., TOS3: TOS2, TOS1: TOS}``.

.. versionchanged:: 3.5
The dictionary is created from stack items instead of creating an
empty dictionary pre-sized to hold *count* items.

opcode:: BUILD_TUPLE (count)
Creates a tuple consuming *count* items from the stack, and pushes the
resulting tuple onto the stack.

opcode:: BUILD_LIST (count)
Works as :opcode:`BUILD_TUPLE`, but creates a list.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45


TARGET(BUILD_MAP) {
Py_ssize_t i;
PyObject *map = _PyDict_NewPresized((Py_ssize_t)oparg);
if (map == NULL)
goto error;
for (i = oparg; i > 0; i--) {
int err;
PyObject *key = PEEK(2*i); // 偶数
PyObject *value = PEEK(2*i - 1); // 奇数
err = PyDict_SetItem(map, key, value);
if (err != 0) {
Py_DECREF(map);
goto error;
}
}

while (oparg--) {
// 这里挺有趣,每次都 POP 两个
Py_DECREF(POP());
Py_DECREF(POP());
/* 从上面的偶数,奇数,也不难猜出。
字典 k-v 都是通过 LOAD_CONST,压入了栈中
比较奇特的是:d = {"X": "Z", 'a': 'b'},结果是 key 在一起
>>> co.co_consts == ('Z', 'b', ('X', 'a'), None)
字节码对应 BUILD_CONST_KEY_MAP
*/
}
PUSH(map);
DISPATCH();
}

TARGET(BUILD_LIST) {
PyObject *list = PyList_New(oparg);
if (list == NULL)
goto error;
while (--oparg >= 0) {
// 同样,列表中的值,也是在栈中,依此读取
PyObject *item = POP();
PyList_SET_ITEM(list, oparg, item);
}
PUSH(list);
DISPATCH();
}

RETURN_VALUE

1
2
3
4
5
6
7
8
              8 LOAD_CONST               1 (None)
10 RETURN_VALUE

TARGET(RETURN_VALUE) {
retval = POP();
why = WHY_RETURN; // 0x0008, /* 'return' statement */
goto fast_block_end;
}

最后,临走前将返回值 None 压入栈中,然后在 POP 出来,break 掉死循环。

BINARY_ADD

1
2
3
4
a = 1 + 3

co_consts: (1, 3, None, 4)
co_names: ('a',)

这种,在编译时直接就计算了结果。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
a = 1
b = 2
c = 3
d = a + b * (c + c)

BINARY_ADD:TOS = TOS1 + TOS

4 12 LOAD_NAME 0 (a)
14 LOAD_NAME 1 (b)
16 LOAD_NAME 2 (c)
18 LOAD_NAME 2 (c)
20 BINARY_ADD
22 BINARY_MULTIPLY
24 BINARY_ADD
26 STORE_NAME 3 (d)

这种,就是正常的,先调用LOAD_NAME在调用BINARY_ADD。而且,比较重要的是,只能每次两个两个相加,并且自左向右结合,优先级在编译时考虑。

LOAD_NAME

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// names = f->co->co_names,可见是从 符号表中加载

TARGET(LOAD_NAME) {
PyObject *name = GETITEM(names, oparg);
PyObject *locals = f->f_locals;
PyObject *v;

v = PyObject_GetItem(locals, name);

if (v == NULL) {
v = PyDict_GetItem(f->f_globals, name);
Py_XINCREF(v);
if (v == NULL) {
v = PyDict_GetItem(f->f_builtins, name);
if (v == NULL) {
goto error;
}
Py_INCREF(v);
}
}
PUSH(v);
DISPATCH();
}

精简代码如上,很好理解,从 locals -> globals -> builtins,找到了,就压栈,找不到就报错。