Official compatibility: Odoo 11 will be the first LTS release to introduce Python 3 compatibility, starting with Python 3.5. It will also be the first LTS release to drop official support for Python 2.
Rationale: Python 3 has been around since 2008, and all Python libraries used by the official Odoo distribution have been ported and are considered stable. Most supported platforms have a Python 3.5 package, or a similar way to deploy it. Preserving dual compatibility is therefore considered unnecessary, and would represent a significant overhead in testing for the lifetime of Odoo 11.
Python 2 and Python 3 are somewhat different language, but following backports, forward ports and cross-compatibility library it is possible to use a subset of Python 2 and Python 3 in order to have a system compatible with both.
Here are a few useful steps or reminders to make Python 2 code compatible with Python 3.
This is not a general-purpose guide for porting Python 2 to Python 3, it’s a guide to write 2/3-compatible Odoo code. It does not go through all the changes in Python but rather through issues which have been found in the standard Odoo distribution in order to show how to evolve such code such that it works on both Python 2 and Python 3.
- What’s new in Python 3? covers many of the changes between Python 2 and Python 3, though it is missing a number of changes which were backported to Python 2.7 as well as some feature reintroductions of later Python 3 revisions
- How do I port to Python 3?
- Porting Python 2 code to Python 3
- Porting to Python 3: A Guide (a bit outdated but useful for the extensive comments on strings and IO)
A cross compatible Odoo would only support Python 2.7 and Python 3.5 and above: Python 2.7 backported some Python 3 features, and Python 2 features were reintroduced in various Python 3 in order to make conversion easier. Python 3.6 adds great features (f-strings, …) and performance improvements (ordered compact dicts) but does not seem to reintroduce compatibility features whereas:
- Python 3.5 reintroduced
%for bytes/bytestrings (PEP 461)
- Python 3.4 has no specific compatibility improvement but is the lowest P3 version for PyLint
- Python 3.3 reintroduced the “u” prefix for proper (unicode) strings
- Python 3.2 made
rangeviews more list-like (backported to 2.7)and reintroduced
Dict & set iteration order (“Hash Randomisation”)
In Python 2, the iteration order depends on the value’s hash (modulo the
collection’s capacity and conflict resolution), which provides a
spec-undefined but implementation-defined order. While that’s not supposed to
happen, it turns out code may depend on the specific order of iteration over
a hash collection (
Python 3.3 enables hash randomisation by default (this can be optionally
enabled on previous versions including Python 2 by providing the
command-line parameter), which means the order of iteration changes from one
run to the next.
When discovered, this can be fixed by one of:
- making iteration steps properly independent (removing the dependency of order of iteration)
- using different checking method (e.g. when serialising sets or dictionaries and checking against the specific serialised value)
- fixing dependencies
- using a
odoo.tools.misc.OrderedSetinstead of a regular one, they guarantee order of iteration is order of insertion
- sorting the collection’s items before iterating over them (this may require adding some sort of iteration key to the items)
Moved and removed
Standard Library Modules
Python 3 reorganised, moved or removed a number of modules in the standard library:
cStringIOwere removed, you can use
io.StringIOto replace them in a cross-version manner (
io.BytesIOfor binary data,
io.StringIOfor text/unicode data).
urlparsewere redistributed across
Since requests and werkzeug are already hard dependencies of Odoo, replace
urllib2.Requestuses by requests, and
urlparseand a few utilty functions (
urllib.urlencode) are available through
werkzeug.urls, a backport of Python 3’s
requests does not raise by default on non-200 responses
cgi.escape(HTML escaping) is deprecated in Python 3, prefer Odoo’s own
- Most of
types’s content has been stripped out in Python 3: only “internal” interpreter types (e.g. CodeType, FrameType, …) have been left in, other types can be obtained directly from the corresponding builtin or by getting the
type()of a literal value.
Absolute Imports (PEP 328)
In Python 3,
import foo can only import from a “top-level” library
(absolute path). If trying to import a sibling or sub-module you must
use an explicitly relative import e.g.
from . import foo or
from .foo import bar.
In Python 2
import statements are ambiguous: if a file
import b, the import system will first check if there’s a
next to it before checking if there is a package called that on the
Furthermore if a sibling file is named the same as top-level package, the
library becomes inaccessible to both the file itself ans siblings, this has
actually happened in Odoo with
Additionally, relative imports allow navigating “up” the tree by using
Explicitly relative imports are always available in Python 2, and should be used everywhere.
You can ensure you are not using any implicitly relative import by adding
from __future__ import absolute_import at the top of your files, or by
All exception handlers must be converted to
except ... as ... Valid
except Exception: except (Exception1, ...): except Exception as name: except (Exception1, ...) as name:
In Python 2,
except statements are of the form:
except Exception[, name]:
except (Exception1, Exception2)[, name]:
But because the name is optional, this gets confusing and people can stumble into the first form when trying for the second and write:
except Exception1, Exception:
which will not yield the expected result.
Python 3 changes this syntax to:
except Exception[ as name]:
except (Exception1, Exception2)[ as name]:
This form was implemented in Python 2.5 and is thus compatible across the board.
Operators & keywords
The backtick operator
`foo` must be converted to an
explicit call to the
<> operator must be replaced by
These two operators were long recommended against/deprecated in Python 2, Python 3 removed them from the language.
exec is now a builtin
In Python 2,
exec is a statement/keyword. Much like
statement to the following cross-language forms:
exec(source) exec(source, globals) exec(source, globals, locals)
List/iteration builtins and methods
In Python 3, a number of builtins and methods formerly returning lists were converted to return iterators or views, with the corresponding redundant methods or functions having been removed entirely:
In Python 3,
itertools.iziphave been removed.
When possible, use comprehensions (list, generator, …) rather than
In Python 3,
dict.itemsreturn views rather than lists, and the
view*methods have been removed.
When the result of the above methods is used for more than a one-shot loop (e.g. to be included in returned value), or when the dict needs to be modified during iteration, wrap the calls in a
cmp builtin function has been removed from Python 3.
- Most of its uses are in
cmp=parameters to sort functions where it can usually be replaced by a key function.
- Other uses found were obtaining the sign of an item (
cmp(item, 0)), this can be replicated using the standard library’s
math.copysign(1, item)will return
itemis positive and
execfile(path) has been removed completely from Python 3 but it is
trivially replaceable in all cases by:
of a variant thereof (see exec changes for details)
file builtin has been removed in Python 3. Generally, it can just
be replaced by the
open builtin, although you may want to use
which is more flexible and better handles the binary/text dichotomy,
a big issue in cross-version Python.
In Python 3, the
open builtin is actually an alias for
In Python 2, integers can be either
long. Python 3 unifies this
under the single
Lsuffix for integer literals must be removed
- calls to
longmust be replaced by calls to
Lsuffix on numbers is unsupported in Python 3, and unnecessary in Python 2 as “overflowing” integer literals will implicitly instantiate long.
- in Python 2, a call to
int()will implicitly create a
longobject if necessary.
- type-testing is the last and bigger issue as in Python 2
longis not a subtype of
int(nor the reverse), and
isinstance(value, (int, long))is thus generally necessary to catch all integrals.
In Python 3,
reduce has been demoted from builtin to
However this is because most uses of ``reduce`` can be replaced by ``sum``,
``all``, ``any`` or a list comprehension for a more readable and faster
It is easy enough to just add
from functools import reduce to the file
and compatible with Python 2.6 and later, but consider whether you get better
code by replacing it with some other method altogether.
In Python 3,
range() behaves the same as Python 2’s
For cross-version code, you can just use
range() everywhere: while this
will incur a slight allocation cost on Python 2, Python 3’s
the entire Sequence protocol and thus behaves very much like a regular
list or tuple.
has_keymethod on dicts must be replaced by use of the
bar in foo.
in for dicts was introduced in Python 2.3, leading to
redundant, and removed in Python 3.
Minor syntax changes
the ability to unpack a parameter (in the parameter declaration list) has been removed in Python 3 e.g.:
def foo((bar, baz), qux): …
is now invalid
octal literals must be prefixed by
0O). Following the C family, in Python 2 an octal literal simply has a leading 0, which can be confusing and easy to get wrong when e.g. padding for readability (e.g.
0013would be the decimal 11 rather than 13).
In Python 3, leading zeroes followed by neither a 0 nor a period is an error, octal literals now follow the hexadecimal convention with a
Bytes/String/Text: The Big One
The most impactful Python 3 change by far is to the text model: for historical
reasons the distinction Python 2’s bytestrings (
str) and text
unicode) is fuzzy and it will try to implicitly convert between
one and the other using the ASCII encoding.
Python 3 changes this, it removes the implicit conversions, removes APIs which contribute to the fuzz and tends to strictly segregate other to work on either bytes or text.
This is fundamentally good and mostly sensible, but it means lots of breakage:
Python 3 removes both
corresponds to text strings (the old
bytestrings in both languages 1.
Both versions have the following prefixes for string literals:
b'foo'is a bytestring (
'foo'is that version’s
strtype, which may be either a bytestring or a text string 2.
u'foo'is that version’s text string.
For best cross-version compatibility you should avoid unprefixed string literals unless you specifically need a “native string” 2.
open builtin should always be explicitly used in binary mode
To read text files, use
On both P2 and P3,
open defaults to returning native strings in default
(“text”) mode, however in P3 that means it actually decodes the file’s bytes
using whatever encoding was set up (default: UTF-8) while on Python 2 it has
no concept of encoding.
open in binary mode provides bytestrings on both versions and works
fine. To read text files, use
io.open and provide an explicit encoding.
base64 is a bytes->bytes conversion. bytes->bytes codecs were removed from the “native” encoding/decoding system which is now exclusively for bytes<->text conversions: text is encoded to bytes and bytes are decoded to text.
bytes.decode('base64') must be
migrated to using
csv is a fairly vicious one: not only is it not a very good format, the
Python 2 and Python 3 versions of the library are text-model incompatible in
- Python 2’s CSV only works on ascii-compatible byte streams (it has no encoding support at all) and extracts bytestring values
- Python 3’s CSV only works on text streams and extract text values
iodoesn’t provide “native string” streaming facilities.
However with respect to Odoo it turns out most or all uses of
inside a model of byte stream to and from text values.
The latter is thus a model implemented by cross-version wrappers
odoo.tools.pycompat.csv_writer(): they take a UTF-8 byte stream and
read or write text values.
b"f", but in Python 3 it’s
102(the value of the first byte), you’ll want to slice bytestrings for compatibility.
csvmodule of the standard library is one such problematic API (it is also notoriously problematic for its terrible support of non-ascii-compatible encodings in Python 2).
email.message_from_stringis an other one.