Pyste & Py++ comparison¶

What is Pyste?¶

Pyste is a Boost.Python code generator. The user specifies the classes and functions to be exported using a simple interface file, which following the Boost.Python’s philosophy, is simple Python code. Pyste then uses GCC-XML to parse all the headers and extract the necessary information to automatically generate C++ code.

Preamble¶

If you are reading this document, I can assume that you know what Boost.Python, Pyste and Py++ are. This document compares Pyste and Py++. I am going to compare:

user interface

design

dependencies

features list:

supported Boost.Python components

nice features

I did not develop Pyste, so it is possible I made few mistakes in my analyzes. If you identify one, please report it. I will try to fix it, as quick as possible. In the past, I created bindings to few projects using Pyste. Code, generated by Pyste, was pretty good and gave me a good start both with my projects and with Boost.Python library. As for me, there are two main problems with Pyste:

It is time-consuming operation to maintain Pyste scripts in a big, dynamic project.

It is very difficult to customize generated code. I will provide few examples later.

I tried to fix Pyste. At first, I developed pygccxml and tried to replace relevant Pyste functionality. I did not like the result. After this, I dropped the idea to fix Pyste and decided to develop new code generator - Py++. Later you will find few points, which explains, why I decided not to fix Pyste.

Have a nice reading.

User interface¶

Pyste¶

Carefully read Pyste definition -

“… The user specifies the classes and functions to be exported using a simple interface file, which following the Boost.Python’s philosophy, is simple Python code. …”

Simple interface file is an advantage and a disadvantage at the same time. The advantage is obvious: it is easy to start to use. The disadvantage is less obvious - you cannot work on whole exported library at once.

Try to exclude all functions, that nth argument has some specific type.

Try to exclude\leave all classes, which belongs to some namespace.

Maintenance. Every time you add new file\class to your project you should modify\create Pyste interface files.

Try to set call policies to functions based on some detail\fact.

First and second example could be solved, but it is neither easy, nor obvious. You should understand Pyste implementation details. Third example is solvable if you are lucky and ready to write few Python\shell scripts.

“… AllFromHeader is broken in some cases. Until it is fixed, use at you own risk. …”

—Pyste documentation.

I have more examples, but I think you’ve got the idea.

Py++¶

Py++ has 2 user interfaces:

GUI - small and simple graphic user interface, which does not request from user any knowledge about Boost.Python or Py++. You can see its screenshot.

API - object-oriented framework, that helps you to create code generator, that suites your needs. Py++ framework consists 3 packages:

Code creators package. The only responsibility of classes in this package is to create code. Those classes do not care where code is going to be written. Neither they care about the order of generated code.

Module creator package. This is code creators factory. Classes in this package, analyze C++ declarations and creates relevant code creators.

File writers package. This package contains classes that write generated code to file(s).

The good news: it is very easy to evaluate Py++, using GUI. The bad news is that you should learn another set of API’s in order to complete your task. It is not as awful as it sounds, but still there is some learning curve.

Design¶

Pyste¶

Pyste design is not bad at all, but there were few mistakes made. First of all current situation. There is a class hierarchy that represents C++ declarations and types. There is Exporter’s hierarchy. There are 2 unrelated classes SingleCodeUnit and MultipleCodeUnit.

Class Exporter has few responsibilities:

to create code

to write code into code unit

to decide what, how and in which order declarations should be exported (excluded)

Classes SingleCodeUnit and MultipleCodeUnit also has few responsibilities:

to write code into files

to control the order of written code

Py++¶

One of the biggest weaknesses of Pyste is a lack of good GCC-XML front-end and this fact cause Pyste to generate not optimal code. I will prove this later. In order to avoid such weakness, before I have created Py++, I created pygccxml. pygccxml is a stand-alone project, which provides few things:

hierarchy of C++ declarations

hierarchy of C++ types

few parsing strategies:

all files will be parsed as it was one file that includes all them

every file will be parsed alone, after this, duplicated declarations and types will be removed

pygccxml contributes in few ways to Py++:

Py++ has nothing to do with code parsing. Theoretically another back-end could be added to pygccxml without changing even one line of code within Py++.
pygccxml has type traits. A lot of type traits algorithms from boost.type_traits library has been implemented. Py++ makes an extensive use of them:
identify “call policies” algorithm
code creator that generates body of “overridden” virtual functions:
struct expensive_to_copy{...};
void do_smth( expensive_to_copy* x, const expensive_to_copy& y ){...}
void
do_smth(expensive_to_copy* x, const expensive_to_copy& y){
    //Pyste generates the following code
    //call_method< void >(self, "do_smth", x, y);
    //Py++ generates the following code
    *this->get_override("do_smth")( boost::python::ptr(x), boost::ref(y) );
    //------------------------------^^^^^^^^^^^^^^^^^^-----^^^^^^^^^^
}
Do you see the difference? This is a prove to the point I made earlier: Pyste generates not optimal code.
Do you remember first 3 problems, I talked about?

Try to exclude all functions, that nth argument has some specific type.

Try to exclude\leave all classes, which belongs to some namespace.

Maintenance. Every time you add new file\class to your project you should modify\create Pyste interface files.

They are solved by pygccxml package. Please take a look on pygccxml query interface documentation.

Now about mistakes. There is a gold rule: one class, one responsibility. Pyste breaks this rule. One more time Exporter class responsibilities:

to create code

to write code into code unit

to decide what, how and in which order declarations should be exported (excluded)

What is wrong with this? In some places the logic of code generation is complex. Pyste makes this situation worse. In order to understand my point try to see, what is going on, in “ClassExporter.py” file. Class ClassExporter knows everything about internal exporters, more over ClassExporter is responsible to export all its base classes before it exports itself. This is just wrong. Py++ implements better solution. Py++ has code creators class hierarchy and factory, that creates those code creators based on given declarations. The questions like:

what should be the order of exporting declarations

could \ should the declaration be exported

what are the call policies of the function

what is the class held type

solved by the factory. The only thing that left to code creators is to create code. By the way, it is not an easy task to fix this specific problem in Pyste. Serious re-factoring should be done. Py++ defines compound_t code creator. module_t, module_body_t, class_t - are typical examples of compound code creators. They do know few code creators, but that is all.

Another weakness is a relationship between Exporter’s and code units. In order to write code into code unit, Exporter should specify section name and text. There are few problems with this. First of all, this tight coupling is unnecessary. In order to add new Exporter you should understand the underlying protocol and may be to modify code unit. Try to add Exporter that exposes get\set functions as property. It is not an easy task. Another problem, I see here, is that code unit is some kind of Exporter. Code unit also creates code. I did not fill good with this design. So I decided, that in Py++ code should be created only by code creators. This decision simplifies a lot of things:

code creators does not depend on file writers

file writers do not have to know all code creators

code creators set is not closed and could be extended

user can place custom code creator almost anywhere

code creators can take into account code that is going to be generated by other code creators. For example namespace aliases or include directives.

…

Dependencies¶

Both Pyste and Py++ introduce one external dependency. In order to parse XML Pyste uses elementtree. On the other side Py++ depends on pygccxml project.

Features list¶

Py++ supports almost all features Pyste implements. Py++, version 0.8.0, does not implements the following functionality, implemented by Pyste:

pow operator

good documentation

Here you can find the complete features list of Py++.

Features unique to Py++:

Py++ exposes protected member functions

Py++, in most cases, will automatically detect class held type

Py++ creates implicit conversion code for:

casting constructors

casting operators

smart pointers of derived and base class

class wrappers generated using Boost.Python wrapper class

operator() always exposed ( this is not the case with Pyste )

operators defined in base class could be redefined/exposed in derived class

Py++ exposes bit fields member variables

Nice features list¶

Both Pyste and Py++ generate working code. As we already saw in some cases Py++ do better job. Py++ allows the following customization on generated code:

To define std and user directories. include_t code creator will take those directories into account.
To define namespace alias.
namespace dt = boost::date_time;
All code, which is generated after this statement, will use dt instead of boost::date_time. This allows Py++ to create user-friendly code.
Classes and functions support 2 modes of code generation. Example:
struct world{
    void set(std::string msg) { this->msg = msg; }
    std::string greet() { return msg; }
    std::string msg;
};
First mode:
class_<world>("world")
   .def("greet", &world::greet)
   .def("set", &world::set)
;
Second mode:
if( true ){
    typedef class_<world> world_exposer_t;
    world_exposer_t world_exposer;
    boost::python::scope world_scope( exposer );
    world_exposer.def( "greet", ( std::string ( world::* )() )( &world::greet ) );
    world_exposer.def( "set", ( void ( world::* )(std::string) )( &world::set ) );
}
Second mode is better then first, because:

It is easier to understand compilation error.

If in future a developer decides to create overload to some function, this code will continue to compile.
Py++ has small nice future - “license”. User can specify the license and it will appear in every generated file.

Py++ allows user to define custom call policies resolver. See boost.date_time example

Py++ allows user to create custom code creators. See “custom_code_creator” example.

real world examples:

EasyBMP

boost.date_time

Qt.XML

Conclusion¶

If I were you I would choose Py++ to create bindings for your project. For very small projects or for educational reasons you may use Py++.GUI. For big projects, you need flexibility and power of Py++.

P.S.¶

This comparison was a little unfair. First of all Pyste is no more under active development\support. Second, Pyste has been written 2 years ago and had different goal.

Pyste and Py++ have been created to handle different tasks, hence the difference in design, user interface and complexity. Bruno da Silva de Oliveira, the Pyste author, understands the problems, I raised here. He thought about them, long before I created Py++. But unfortunately, lack of time and motivation prevents him to work on Pyste.