Softpedia >Mac >Developer Tools >PyPy >  Changelog

PyPy Changelog

What's new in PyPy 5.3.1

Jun 22, 2016
  • A bug-fix release, merging these changes:
  • Add include guards to pymem.h, fixes issue #2321
  • Make vmprof build on OpenBSD, from pull request #456
  • Fix bytearray('').replace('a', 'ab'), issue #2324

New in PyPy 4.0.1 (Dec 20, 2015)

  • Bug Fixes:
  • Fix a bug when unrolling double loops in JITted code
  • Fix multiple memory leaks in the ssl module, one of which affected cpython as well (thanks to Alex Gaynor for pointing those out)
  • Use pkg-config to find ssl headers on OS-X
  • Issues reported with our previous release were resolved after reports from users on our issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at #pypy
  • New features:
  • Internal cleanup of RPython class handling
  • Support stackless and greenlets on PPC machines
  • Improve debug logging in subprocesses: use PYPYLOG=jit:log.%d for example to have all subprocesses write the JIT log to a file called ‘log.%d’, with ‘%d’ replaced with the subprocess’ PID.
  • Support PyOS_double_to_string in our cpyext capi compatibility layer
  • Numpy:
  • Improve support for __array_interface__
  • Propagate NAN mantissas through float16-float32-float64 conversions
  • Performance improvements and refactorings:
  • Improvements in slicing byte arrays
  • Improvements in enumerate()
  • Silence some warnings while translating

New in PyPy 4.0.0 (Oct 30, 2015)

  • NEW VERSION NUMBERING:
  • Since the past release, PyPy 2.6.1, we decided to update the PyPy 2.x.x versioning directly to PyPy 4.x.x, to avoid confusion with CPython 2.7 and 3.5. Note that this version of PyPy uses the stdlib and implements the syntax of CPython 2.7.10.
  • VECTORIZATION:
  • Richard Plangger began work in March and continued over a Google Summer of Code to add a vectorization step to the trace optimizer. The step recognizes common constructs and emits SIMD code where possible, much as any modern compiler does. This vectorization happens while tracing running code, so it is actually easier at run-time to determine the availability of possible vectorization than it is for ahead-of-time compilers.
  • Availability of SIMD hardware is detected at run time, without needing to precompile various code paths into the executable.
  • The first version of the vectorization has been merged in this release, since it is so new it is off by default. To enable the vectorization in built-in JIT drivers (like numpy ufuncs), add –jit vec=1, to enable all implemented vectorization add –jit vec_all=1
  • Internal Refactoring: Warmup Time Improvement and Reduced Memory Usage:
  • Maciej Fijalkowski and Armin Rigo refactored internals of rpython that now allow PyPy to more efficiently use guards in jitted code. They also rewrote unrolling, leading to a warmup time improvement of 20% or so. The reduction in guards also means a reduction in the use of memory, also a savings of around 20%.
  • Numpy:
  • Our implementation of numpy continues to improve. ndarray and the numeric dtypes are very close to feature-complete; record, string and unicode dtypes are mostly supported. We have reimplemented numpy linalg, random and fft as cffi-1.0 modules that call out to the same underlying libraries that upstream numpy uses. Please try it out, especially using the new vectorization (via –jit vec=1 on the command line) and let us know what is missing for your code.
  • CFFI:
  • While not applicable only to PyPy, cffi is arguably our most significant contribution to the python ecosystem. Armin Rigo continued improving it, and PyPy reaps the benefits of cffi-1.3: improved manangement of object lifetimes, __stdcall on Win32, ffi.memmove(), and percolate const, restrict keywords from cdef to C code.
  • BUG FIXES:
  • Applied OPENBSD downstream fixes
  • Fix a crash on non-linux when running more than 20 threads
  • In cffi, ffi.new_handle() is more cpython compliant
  • Accept unicode in functions inside the _curses cffi backend exactly like cpython
  • Fix a segfault in itertools.islice()
  • Use gcrootfinder=shadowstack by default, asmgcc on linux only
  • Fix ndarray.copy() for upstream compatability when copying non-contiguous arrays
  • Fix assumption that lltype.UniChar is unsigned
  • Fix a subtle bug with stacklets on shadowstack
  • Improve support for the cpython capi in cpyext (our capi compatibility layer). Fixing these issues inspired some thought about cpyext in general, stay tuned for more improvements
  • When loading dynamic libraries, in case of a certain loading error, retry loading the library assuming it is actually a linker script, like on Arch and Gentoo
  • Issues reported with our previous release were resolved after reports from users on our issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at #pypy
  • NEW FEATURES:
  • Add an optimization pass to vectorize loops using x86 SIMD intrinsics.
  • Support __stdcall on Windows in CFFI
  • Improve debug logging when using PYPYLOG=???
  • Deal with platforms with no RAND_egd() in OpenSSL
  • NUMPY:
  • Add support for ndarray.ctypes
  • Fast path for mixing numpy scalars and floats
  • Add support for creating Fortran-ordered ndarrays
  • Fix casting failures in linalg (by extending ufunc casting)
  • Recognize and disallow (for now) pickling of ndarrays with objects embedded in them
  • PERFORMANCE IMPROVEMENTS AND REFACTORINGS:
  • Reuse hashed keys across dictionaries and sets
  • Refactor JIT interals to improve warmup time by 20% or so at the cost of a minor regression in JIT speed
  • Recognize patterns of common sequences in the JIT backends and optimize them
  • Make the garbage collecter more incremental over external_malloc() calls
  • Share guard resume data where possible which reduces memory usage
  • Fast path for zip(list, list)
  • Reduce the number of checks in the JIT for lst[a:]
  • Move the non-optimizable part of callbacks outside the JIT
  • Factor in field immutability when invalidating heap information
  • Unroll itertools.izip_longest() with two sequences
  • Minor optimizations after analyzing output from vmprof and trace logs
  • Remove many class attributes in rpython classes
  • Handle getfield_gc_pure* and getfield_gc_* uniformly in heap.py
  • Improve simple trace function performance by lazily calling fast2locals and locals2fast only if truly necessary

New in PyPy 2.6.0 (Jun 23, 2015)

  • Python compatibility:
  • Improve support for TLS 1.1 and 1.2
  • Windows downloads now package a pypyw.exe in addition to pypy.exe
  • Support for the PYTHONOPTIMIZE environment variable (impacting builtin’s __debug__ property)
  • Issues reported with our previous release were resolved after reports from users on our issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at #pypy.
  • New features:
  • Add preliminary support for a new lightweight statistical profiler vmprof, which has been designed to accomodate profiling JITted code
  • Numpy:
  • Support for object dtype via a garbage collector hook
  • Support for .can_cast and .min_scalar_type as well as beginning a refactoring of the internal casting rules
  • Better support for subtypes, via the __array_interface__, __array_priority__, and __array_wrap__ methods (still a work-in-progress)
  • Better support for ndarray.flags
  • Performance improvements:
  • Slight improvement in frame sizes, improving some benchmarks
  • Internal refactoring and cleanups leading to improved JIT performance
  • Improved IO performance of zlib and bz2 modules
  • We continue to improve the JIT’s optimizations. Our benchmark suite is now over 7 times faster than cpython

New in PyPy 2.5.1 (Mar 26, 2015)

  • HIGHLIGHTS:
  • We merged version 2.7.9 of python’s stdlib. From the python release notice:
  • The entirety of Python 3.4’s ssl module has been backported. See PEP 466 for justification.
  • HTTPS certificate validation using the system’s certificate store is now enabled by default. See PEP 476 for details.
  • SSLv3 has been disabled by default in httplib and its reverse dependencies due to the POODLE attack.
  • The ensurepip module has been backported, which provides the pip package manager in every Python 2.7 installation. See PEP 477.
  • The garbage collector now ignores parts of the stack which did not change since the last collection, another performance boost
  • errno and LastError are saved around cffi calls so things like pdb will not overwrite it
  • We continue to asymptotically approach a score of 7 times faster than cpython on our benchmark suite, we now rank 6.98 on latest runs
  • FXES:
  • Non-blocking file reads sometimes raised EAGAIN even though they had buffered data waiting, fixed in b1c4fcb04a42
  • Fix a bug in cpyext in multithreded programs acquiring/releasing the GIL
  • Avoid tracing all stack roots during repeated minor collections, by ignoring the part of the stack that didn’t change
  • Update stdlib to version 2.7.9
  • Fix exception being raised by kqueue.control (CPython compatibility)
  • Refactor rpython.flowspace.framestate.FrameState.
  • Add an alternative location to save LastError, errno around ctypes, cffi external calls so things like pdb will not overwrite it
  • Speed up the warmup times of the JIT by removing a quadratic algorithm in the heapcache.
  • Simplify flow graphs on the fly during annotation phase.

New in PyPy 2.5.0 (Feb 4, 2015)

  • Highlights:
  • The past months have seen pypy mature and grow, as rpython becomes the goto solution for writing fast dynamic language interpreters. Our separation of rpython and the python interpreter PyPy is now much clearer in the PyPy documentation and we now have seperate RPython documentation.
  • We have improved warmup time as well as jitted code performance: more than 10% compared to pypy-2.4.0, due to internal cleanup and gc nursery improvements. We no longer zero-out memory allocated in the gc nursery by default, work that was started during a GSoC.
  • Passing objects between C and PyPy has been improved. We are now able to pass raw pointers to C (without copying) using pinning. This improves I/O; benchmarks that use networking intensively improved by about 50%. File() operations still need some refactoring but are already showing a 20% improvement on our benchmarks. Let us know if you see similar improvements.
  • Our integrated numpy support gained much of the GenericUfunc api in order to support the lapack/blas linalg module of numpy. This dovetails with work in the pypy/numpy repository to support linalg both through the (slower) cpyext capi interface and also via (the faster) pure python cffi interface, using an extended frompyfunc() api. We will soon post a seperate blog post specifically about linalg and PyPy.
  • Dictionaries are now ordered by default, see the blog post
  • Our nightly translations use –shared by default, including on OS/X and linux
  • We now more carefully handle errno (and GetLastError, WSAGetLastError) tying the handlers as close as possible to the external function call, in non-jitted as well as jitted code.
  • Issues reported with our previous release were resolved after reports from users on our issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at #pypy.

New in PyPy 2.4.0 (Sep 22, 2014)

  • Benchmarks improved after internal enhancements in string and bytearray handling, and a major rewrite of the GIL handling. This means that external calls are now a lot faster, especially the CFFI ones. It also means better performance in a lot of corner cases with handling strings or bytearrays. The main bugfix is handling of many socket objects in your program which in the long run used to “leak” memory.
  • PyPy now uses Python 2.7.8 standard library.
  • We fixed a memory leak in IO in the sandbox code
  • We welcomed more than 12 new contributors, and conducted two Google Summer of Code projects, as well as other student projects not directly related to Summer of Code.
  • Issues reported with our previous release were fixed after reports from users on our new issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at #pypy.
  • Here is a summary of the user-facing changes; for more information see whats-new:
  • Reduced internal copying of bytearray operations
  • Tweak the internal structure of StringBuilder to speed up large string handling, which becomes advantageous on large programs at the cost of slightly slower small benchmark type programs.
  • Boost performance of thread-local variables in both unjitted and jitted code, this mostly affects errno handling on linux, which makes external calls faster.
  • Move to a mixed polling and mutex GIL model that make mutlithreaded jitted code run much faster
  • Optimize errno handling in linux (x86 and x86-64 only)
  • Remove ctypes pythonapi and ctypes.PyDLL, which never worked on PyPy
  • Fix performance regression on ufunc(, ) in numpy
  • Classes in the ast module are now distinct from structures used by the compiler, which simplifies and speeds up translation of our source code to the PyPy binary interpreter
  • Upgrade stdlib from 2.7.5 to 2.7.8
  • Win32 now links statically to zlib, expat, bzip, and openssl-1.0.1i. No more missing DLLs
  • Many issues were resolved since the 2.3.1 release on June 8

New in PyPy 2.4 Beta 1 (Sep 9, 2014)

  • Benchmarks improved after internal enhancements in string and bytearray handling, and a major rewrite of the GIL handling. This means that external calls are now a lot faster, especially the CFFI ones. It also means better performance in a lot of corner cases with handling strings or bytearrays. The main bugfix is handling of many socket objects in your program which in the long run used to “leak” memory.
  • PyPy now uses Python 2.7.8 standard library.
  • User-facing changes:
  • Reduced internal copying of bytearray operations
  • Tweak the internal structure of StringBuilder to speed up large string handling, which becomes advantageous on large programs at the cost of slightly slower small benchmark type programs.
  • Boost performance of thread-local variables in both unjitted and jitted code, this mostly affects errno handling on linux, which makes external calls faster.
  • Move to a mixed polling and mutex GIL model that make mutli-threaded jitted code run much faster
  • Optimize errno handling in linux (x86 and x86-64 only)
  • Remove ctypes pythonapi and ctypes.PyDLL, which never worked on PyPy
  • Fix performance regression on ufunc(, ) in numpy
  • Classes in the ast module are now distinct from structures used by the compiler, which simplifies and speeds up translation of our source code to the PyPy binary interpreter
  • Upgrade stdlib from 2.7.5 to 2.7.8
  • Win32 now links statically to zlib, expat, bzip, and openssl-1.0.1i. No more missing DLLs
  • Many issues were resolved since the 2.3.1 release on June 8

New in PyPy 2.3.1 (Jun 9, 2014)

  • The built-in struct module was renamed to _struct, solving issues with IDLE and other modules.
  • Support for compilation with gcc-4.9
  • A rewrite of packaging.py which produces our downloadable packages to modernize command line argument handling and to document third-party contributions in our LICENSE file
  • A CFFI-based version of the gdbm module is now included in our downloads
  • Many issues were resolved since the 2.3 release

New in PyPy 2.3 (May 9, 2014)

  • Bug fixes:
  • The ARM port no longer crashes on unaligned memory access to floats and doubles, and singlefloats are supported in the JIT.
  • Generators are faster since they now skip unnecessary cleanup
  • A first time contributor simplified JIT traces by adding integer bound propagation in indexing and logical operations.
  • Optimize consecutive dictionary lookups of the same key in a chain
  • Our extensive pre-translation test suite now runs nightly on more platforms
  • Fix issues with reimporting builtin modules
  • Fix an RPython bug with loop-unrolling that appeared in the HippyVM PHP port
  • Support for corner cases on objects with __int__ and __float__ methods
  • Fix multithreaded support for gethostbyname_ex and gethostbyaddr
  • Fix handling of tp_name for type objects
  • New Platforms and Features:
  • Support for OpenBSD
  • Code cleanup: we continue to prune out old and unused code, and to refactor large parts of the codebase. We have separated RPython from the PyPy python interpreter, and RPython is seeing use in other dynamic language projects.
  • Support for precompiled headers in the build process for MSVC
  • Tweak support of errno in cpyext (the PyPy implemenation of the capi)

New in PyPy 2.2.1 (Nov 28, 2013)

  • This is a bug fix release. The most important bugs fixed are:
  • an issue in sockets’ reference counting emulation, showing up notably when using the ssl module and calling makefile().
  • Tkinter support on Windows.
  • If sys.maxunicode==65535 (on Windows and maybe OS/X), the json decoder incorrectly decoded surrogate pairs.
  • some FreeBSD fixes.

New in PyPy 2.2 (Nov 14, 2013)

  • Highlights:
  • Our Garbage Collector is now “incremental”. It should avoid almost all pauses due to a major collection taking place. Previously, it would pause the program (rarely) to walk all live objects, which could take arbitrarily long if your process is using a whole lot of RAM. Now the same work is done in steps. This should make PyPy more responsive, e.g. in games. There are still other pauses, from the GC and the JIT, but they should be on the order of 5 milliseconds each.
  • The JIT counters for hot code were never reset, which meant that a process running for long enough would eventually JIT-compile more and more rarely executed code. Not only is it useless to compile such code, but as more compiled code means more memory used, this gives the impression of a memory leak. This has been tentatively fixed by decreasing the counters from time to time.
  • NumPy has been split: now PyPy only contains the core module, called _numpypy. The numpy module itself has been moved to https://bitbucket.org/pypy/numpy and numpypy disappeared. You need to install NumPy separately with a virtualenv: pip install git+https://bitbucket.org/pypy/numpy.git; or by directly doing git clone https://bitbucket.org/pypy/numpy.git, cd numpy, python setup.py install.
  • non-inlined calls have less overhead
  • Things that use sys.set_trace are now JITted (like coverage)
  • JSON encoding used to be very fast, now decoding is as well
  • various buffer copying methods experience speedups (like list-of-ints to int[] buffer from cffi)
  • We finally wrote (hopefully) all the missing os.xxx() functions, including os.startfile() on Windows and a handful of rare ones on Posix.
  • numpy has a rudimentary C API that cooperates with cpyext

New in PyPy 2.1 (Aug 1, 2013)

  • Highlights:
  • JIT support for ARM, architecture versions 6 and 7, hard- and soft-float ABI
  • Stacklet support for ARM
  • Support for os.statvfs and os.fstatvfs on unix systems
  • Improved logging performance
  • Faster sets for objects
  • Interpreter improvements
  • During packaging, compile the CFFI based TK extension
  • Pickling of numpy arrays and dtypes
  • Subarrays for numpy
  • Bugfixes to numpy
  • Bugfixes to cffi and ctypes
  • Bugfixes to the x86 stacklet support
  • Fixed issue 1533: fix an RPython-level OverflowError for space.float_w(w_big_long_number).
  • Fixed issue 1552: GreenletExit should inherit from BaseException.
  • Fixed issue 1537: numpypy __array_interface__
  • Fixed issue 1238: Writing to an SSL socket in PyPy sometimes failed with a “bad write retry” message.

New in PyPy 2.1.0 Beta 2 (Jul 26, 2013)

  • Support for os.statvfs and os.fstatvfs on unix systems.
  • Fixed issue 1533: fix an RPython-level OverflowError for space.float_w(w_big_long_number).
  • Fixed issue 1552: GreenletExit should inherit from BaseException.
  • Fixed issue 1537: numpypy __array_interface__
  • Fixed issue 1238: Writing to an SSL socket in pypy sometimes failed with a “bad write retry” message.
  • distutils: copy CPython’s implementation of customize_compiler, dont call split on environment variables, honour CFLAGS, CPPFLAGS, LDSHARED and LDFLAGS.
  • During packaging, compile the CFFI tk extension.

New in PyPy 2.1.0 Beta 1 (Jul 11, 2013)

  • Bugfixes to the ARM JIT backend, so that ARM is now an officially supported processor architecture
  • Stacklet support on ARM
  • Interpreter improvements
  • Various numpy improvements
  • Bugfixes to cffi and ctypes
  • Bugfixes to the stacklet support
  • Improved logging performance
  • Faster sets for objects

New in PyPy 2.0.2 (May 21, 2013)

  • It fixes a crash in the JIT when calling external C functions (with ctypes/cffi) in a multithreaded context.

New in PyPy 2.0.1 (May 17, 2013)

  • fix an occasional crash in the JIT that ends in RPython Fatal error: NotImplementedError.
  • id(x) is now always a positive number (except on int/float/long/complex). This fixes an issue in _sqlite.py (mostly for 32-bit Linux).
  • fix crashes of callback-from-C-functions (with cffi) when used together with Stackless features, on asmgcc (i.e. Linux only). Now gevent should work better.
  • work around an eventlet issue with socket._decref_socketios().