1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
|
<!DOCTYPE html PUBLIC "-//w3c//dtd html 4.0 transitional//en">
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Mozilla/4.51 (Macintosh; I; PPC) [Netscape]">
<title>About Pyrex</title></head>
<body>
<center>
<h1>
<hr width="100%">Pyrex</h1></center>
<center><i><font size="+1">A language for writing Python extension modules</font></i>
<hr width="100%"></center>
<h2>
What is Pyrex all about?</h2>
Pyrex is a language specially designed for writing Python extension modules.
It's designed to bridge the gap between the nice, high-level, easy-to-use
world of Python and the messy, low-level world of C.
<p>You may be wondering why anyone would want a special language for this.
Python is really easy to extend using C or C++, isn't it? Why not just
write your extension modules in one of those languages?
</p><p>Well, if you've ever written an extension module for Python, you'll
know that things are not as easy as all that. First of all, there is a
fair bit of boilerplate code to write before you can even get off the ground.
Then you're faced with the problem of converting between Python and C data
types. For the basic types such as numbers and strings this is not too
bad, but anything more elaborate and you're into picking Python objects
apart using the Python/C API calls, which requires you to be meticulous
about maintaining reference counts, checking for errors at every step and
cleaning up properly if anything goes wrong. Any mistakes and you have
a nasty crash that's very difficult to debug.
</p><p>Various tools have been developed to ease some of the burdens of producing
extension code, of which perhaps <a href="http://www.swig.org">SWIG</a>
is the best known. SWIG takes a definition file consisting of a mixture
of C code and specialised declarations, and produces an extension module.
It writes all the boilerplate for you, and in many cases you can use it
without knowing about the Python/C API. But you need to use API calls if
any substantial restructuring of the data is required between Python and
C.
</p><p>What's more, SWIG gives you no help at all if you want to create a new
built-in Python <i>type. </i>It will generate pure-Python classes which
wrap (in a slightly unsafe manner) pointers to C data structures, but creation
of true extension types is outside its scope.
</p><p>Another notable attempt at making it easier to extend Python is <a href="http://pyinline.sourceforge.net/">PyInline</a>
, inspired by a similar facility for Perl. PyInline lets you embed pieces
of C code in the midst of a Python file, and automatically extracts them
and compiles them into an extension. But it only converts the basic types
automatically, and as with SWIG, it doesn't address the creation
of new Python types.
</p><p>Pyrex aims to go far beyond what any of these previous tools provides.
Pyrex deals with the basic types just as easily as SWIG, but it also lets
you write code to convert between arbitrary Python data structures and
arbitrary C data structures, in a simple and natural way, without knowing
<i>anything</i> about the Python/C API. That's right -- <i>nothing at all</i>!
Nor do you have to worry about reference counting or error checking --
it's all taken care of automatically, behind the scenes, just as it is
in interpreted Python code. And what's more, Pyrex lets you define new
<i>built-in</i> Python types just as easily as you can define new classes
in Python.
</p><p>Sound too good to be true? Read on and find out how it's done.
</p><h2>
The Basics of Pyrex</h2>
The fundamental nature of Pyrex can be summed up as follows: <b>Pyrex is
Python with C data types</b>.
<p><i>Pyrex is Python:</i> Almost any piece of Python code is also valid
Pyrex code. (There are a few limitations, but this approximation will serve
for now.) The Pyrex compiler will convert it into C code which makes equivalent
calls to the Python/C API. In this respect, Pyrex is similar to the former
Python2C project (to which I would supply a reference except that it no
longer seems to exist).
</p><p><i>...with C data types.</i> But Pyrex is much more than that, because
parameters and variables can be declared to have C data types. Code which
manipulates Python values and C values can be freely intermixed, with conversions
occurring automatically wherever possible. Reference count maintenance
and error checking of Python operations is also automatic, and the full
power of Python's exception handling facilities, including the try-except
and try-finally statements, is available to you -- even in the midst of
manipulating C data.
</p><p>Here's a small example showing some of what can be done. It's a routine
for finding prime numbers. You tell it how many primes you want, and it
returns them as a Python list.
</p><blockquote><b><tt><font size="+1">primes.pyx</font></tt></b></blockquote>
<blockquote>
<pre> 1 def primes(int kmax):<br> 2 cdef int n, k, i<br> 3 cdef int p[1000]<br> 4 result = []<br> 5 if kmax > 1000:<br> 6 kmax = 1000<br> 7 k = 0<br> 8 n = 2<br> 9 while k < kmax:<br>10 i = 0<br>11 while i < k and n % p[i] <> 0:<br>12 i = i + 1<br>13 if i == k:<br>14 p[k] = n<br>15 k = k + 1<br>16 result.append(n)<br>17 n = n + 1<br>18 return result</pre>
</blockquote>
You'll see that it starts out just like a normal Python function definition,
except that the parameter <b>kmax</b> is declared to be of type <b>int</b>
. This means that the object passed will be converted to a C integer (or
a TypeError will be raised if it can't be).
<p>Lines 2 and 3 use the <b>cdef</b> statement to define some local C variables.
Line 4 creates a Python list which will be used to return the result. You'll
notice that this is done exactly the same way it would be in Python. Because
the variable <b>result</b> hasn't been given a type, it is assumed to hold
a Python object.
</p><p>Lines 7-9 set up for a loop which will test candidate numbers for primeness
until the required number of primes has been found. Lines 11-12, which
try dividing a candidate by all the primes found so far, are of particular
interest. Because no Python objects are referred to, the loop is translated
entirely into C code, and thus runs very fast.
</p><p>When a prime is found, lines 14-15 add it to the p array for fast access
by the testing loop, and line 16 adds it to the result list. Again, you'll
notice that line 16 looks very much like a Python statement, and in fact
it is, with the twist that the C parameter <b>n</b> is automatically converted
to a Python object before being passed to the <b>append</b> method. Finally,
at line 18, a normal Python <b>return</b> statement returns the result
list.
</p><p>Compiling primes.pyx with the Pyrex compiler produces an extension module
which we can try out in the interactive interpreter as follows:
</p><blockquote>
<pre>>>> import primes<br>>>> primes.primes(10)<br>[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]<br>>>></pre>
</blockquote>
See, it works! And if you're curious about how much work Pyrex has saved
you, take a look at the <a href="primes.c">C code generated for this module</a>
.
<h2>
Language Details</h2>
For more about the Pyrex language, see the <a href="LanguageOverview.html">Language
Overview</a> .
<h2>
Future Plans</h2>
Pyrex is not finished. Substantial tasks remaining include:
<ul>
<li>
Support for certain Python language features which are planned but not
yet implemented. See the <a href="Manual/Limitations.html">Limitations</a>
section of the <a href="LanguageOverview.html">Language Overview</a> for a current
list.</li>
</ul>
<ul>
<li>
C++ support. This could be a very big can of worms - careful thought required
before going there.</li>
</ul>
<ul>
<li>
Reading C/C++ header files directly would be very nice, but there are some
severe problems that I will have to find solutions for first, such as what
to do about preprocessor macros. My current thinking is to use a separate
tool to convert .h files into Pyrex declarations, possibly with some manual
intervention.</li>
</ul>
</body></html>
|
