What Level of Object Orientation are you?

I’ve seen a lot of arguments, both in real life and on the internet, about object orientation. One of the problems with these arguments is that people don’t always agree on what object orientation actually is. In my mind, the phrase has three distinct meanings. I don’t claim that any of what follows is the strict academic definition, it is just what I think people are usually talking about when they say “object oriented”.

Simply Using Classes and Methods

Sometimes when people say object orientation, they really just mean using classes and methods. Classes allow us to define composite types and to define functions on those types which we call methods. This isn’t really object orientation. You’ll find user defined composite types in plenty of languages that aren’t object oriented like Haskell. It is nice to be able to define a function on your new type as a method. However the fact that methods can be called only with the appropriate object is really just static typing.

Mutating State

The property that really characterises object orientation for me is mutable state. Object orientation doesn’t really make sense without mutability. Objects allow us to expose an API and abstract away the underlying implementation. This means that the objects we create contain some inner state and expose methods that use and potentially change this data. Many functional languages discourage mutability. For example in F# all variables are immutable by default and you have to use the mutable keyword to make them mutable.

Inheritance and Polymorphism

The final flavour of object orientation is the deepest and the darkest. This is the use of inheritance and polymorphism. This really is unambiguously object orientation by anyone’s definition. Real, old school object orientation will involve vast and complicated hierarchies of classes. You will have to deal with an advanced taxonomy of user defined types.

Inheritance does not only allow for simple code re-use. It allows you to take advantage of polymorphism. This means that at run-time different methods will be called depending on the type of your variable. So you might define a virtual method in your abstract base class, and have a different implementation for that method in each of the subclasses.

This sort of stuff, complex inheritance hierarchies and runtime polymorphism has fallen out of fashion but you will still see a lot of it in enterprise software, particularly at large financial institutions.

The Tricky Little Differences Between C# and C++ – part 1

There are a lot of big differences between C++ and C#. Really, despite their similar syntax, they are completely different languages. C# has garbage collection, C++ has manual memory management. C# is compiled to a special intermediate language that is then just in time compiled by the run time. C++ is compiled to machine code.

But, when you transfer between the two languages you are probably not going to get caught out by obvious things like that. It’s much more likely that the small subtle differences will trip you up. I’m going to cover some of these little differences that I’ve come across.

In this post we’ll be looking at uninitialised strings. Suppose we create a string in C++ without instantiating it directly ourselves and then print its contents. Like this:

string s;
cout << s << endl; 

This will just print an empty line, because the string s is empty. However, if you try the same thing in C#, it won’t work quite the same. If we use C# and do something like:

string s;
Console.WriteLine(s);

we’ll get the following runtime error:

Program.cs(10,31): error CS0165: Use of unassigned local variable 's'

This is a little surprising, normally C# is the more user friendly language. But in this case in C++ we get a friendly default behaviour whereas in C# we get a nasty runtime exception. Why?

This is because, in our C++ example we created a string object on the stack and dealt with it directly. When we created it, C++ called the default constructor which creates an empty string. However, in C# strings are reference types. This means that whenever we create one we are creating an object on the (managed) head. So our C# is really equivalent to the following C++ code:

String* s = NULL;
count << *s << endl;

If you run this you’ll end up with a seg fault, that’s because a null pointer points to memory address 0 which is inaccessible to you.

Don’t Get Caught out by Covariance!

Downcasting is bad, you shouldn’t do it, it’s code smell and it’s an anti-pattern. Unfortunately, in real world code you will see plenty of it. So you need to be aware of the pottential pitfalls of using downcasting. One that caught me out recently is covariance.

In C++, you can quite freely cast between types. You can take a pointer to some memory and cast it to any type you like, and read out what you get. Generally speaking you won’t get anything useful. If you’re not careful you’ll get “undefined behaviour”. If you have data stored in memory, the assumption is that you know that type that data is, and you are responsible for using it appropriately.

In languages like Java and C#, things are different. Here, the runtime checks our casts and will throw an exception if it thinks you have gone wrong. The consequences of this difference can sometimes be surprising.

Let’s look at an example. Suppose we have a base class Security defined like so:

public class Security
{
    int Id;
    public Security(int id)
    {
        Id = id;
    }
}

and two subclasses, Stock:

public class Stock : Security
{
    string Name;
    public Stock(int id, string name) : base(id)
    {
        Name = name;
    }
}

and Bond:

public class Bond : Security
{
    double Rate;
    public Bond(string name, double rate) : base(id)
    {
        Rate = rate;
    }
}

Now if we have a reference of type Security, that actually points to a Stock, we can happily downcast it to a reference of type Stock, like this:

Security s1 = new Stock(1, "GOOG");
Stock AsStock = (Stock)s1;
Console.WriteLine(s1.Name);

Because, Stock is a subtype of Security we can cast a reference to a Stock to a reference to a Security. This works because every Stock will have all the fields of a Security, in the same relative locations in memory.

However, if you have a reference of type Security that points to a Security or a Bond, and try and cast it to a Stock, you’ll have trouble. If we run the following code

Security s1 = new Bond(1, 0.02);
Stock AsStock = (Stock)s1;
Console.WriteLine(s1.Name);

we will see a run time exception of the form:

Unhandled exception. System.InvalidCastException: Unable to cast object of type 'Casting.Bond' to type 'Casting.Stock'.

This makes sense, the runtime knows that the object we have a reference to is not a Stock. It knows that we can’t cast this object to a stock in a sensible way. So the runtime stops us by throwing an exception.

Let’s look at a slightly more complicated example. Suppose, instead of a single object, we had a whole array of them. The following code:

Security[] securities = new Stock[]{new Stock(1, "ABC"), new Stock(2, "DEF")};
Stock[] stocks = (Stock[]) securities; 
Console.WriteLine(stocks[0].Name);

will run happily. We can cast a reference of type Security[] that points to a Stock[], to a reference of type Stock[]. However if we try

Security[] securities = new Security[]{new Stock(1, "ABC"), new Stock(2, "DEF")};
Stock[] stocks = (Stock[]) securities; 
Console.WriteLine(stocks[0].Name);

we will get an InvalidCastException:

Unhandled exception. System.InvalidCastException: Unable to cast object of type 'Casting.Security[]' to type 'Casting.Stock[]'.

This might seem a little surprising. The objects in our array are still actually Stocks. We know that we can cast a reference of type Security that points to a Stock to a reference of type Security. Why can’t we cast the Security[] reference to a Stock[] reference?

It’s a subtle one. When we cast an array reference, we are not casting the objects in the array, we are casting the array itself. So in the first array example, we are casting a reference of type Security[] to a reference of type Stock[]. The runtime knows that the reference actually points to an object of type Stock[], so this is fine. There will only ever be Stock objects in this array. Even though we have a reference of type Security[] pointing to this array, we can’t do something like:

Security[] securities = new Stock[]{new Stock(1, "ABC"), new Stock(2, "DEF")};
securities[0] = new Bond(2, 0.02);

the runtime knows that our array is of type Stock[], to it throws an exception:

Unhandled exception. System.ArrayTypeMismatchException: Attempted to access an element as a type incompatible with the array.

However, in the second example, we have a reference of type Security[] that points to an array of type Security[]. Although this array only contains stocks, the runtime cannot in general say whether that is true or not. Suppose we had done something like this instead:

Security[] securities = new Security[]{new Stock(1, "ABC"), new Stock(2, "DEF")};
securities[0] = new Bond(2, 0.02);
Stock[] stocks = (Stock[]) securities; 

We can of course add a Bond to an array of type Security[], the runtime doesn’t keep track of all this, which is why it has no way of knowing if the securities array really does contain objects of type Stock or something else.

The name of this type of casting is Covariance. Eric Lippert, one of the original designers of C#, has a pretty good blog about it. The really important point is that when we have an array of some base type, even if we are able to downcast the individual members of that Array, the runtime might stop us from downcasting the entire array.

This tripped me up in my day job last week. I was performing a data load that returned an array of type Security[], I knew this array contained only objects of type Stock and so I cast it to Stock[]. I merged this into master, but then our regression tests failed. Thankfully my mistake was caught before making it into prod.

Don’t Use Objects in Python – part 3

So my previous blog posts, were posted on reddit and I got a lot of interesting feedback. I’ve decided to address the issues that were brought up all together in one place, and this is it! I’ve grouped all the different ideas that came up into their own sections below.

This is against community guidelines

Some people were upset that what I recommended was against the community guidelines. They’re correct, it is. However that isn’t really an argument against doing something. Community guidelines and standards aren’t some holy scripture that must always be obeyed. The entire point of my post was that the common usage of object orientation of Python is bad, I doubt there is anyway I could say that without also advocating against the normal Python standards. There is certainly some value in educating people about community standards, but the value of an idea cannot be reduced to how compliant it is with Python coding guidelines.

Dictionaries are Actually Objects

So a couple of people pointed out that internal to python there is inheritance, and that datatypes like Dictionary actually inherit from the Python Object type. Sure, that’s fine, I don’t have a problem with how the internals of Python implement the various data structures. Indeed the internals are overwhelmingly written in C, so it is not at all relevant, when we’re talking about actual Python code.

Objects are Abstractions of Python Dictionaries

Some commenters argued that Objects in Python are abstractions sitting on-top of dictionaries. This just isn’t true. An abstraction hides the details below it. For example, variables are an abstraction that hide the details of how registers and memory access actually work. Because they limit what you see and what you can do, they make using memory a lot easier. Python objects don’t hide any of the details of a dictionary, they are just syntactic sugar, all the operations of a normal python dictionary are available right their on the object with slightly different syntax, you can even access the dictionary itself with the __dict__ attribute. So this isn’t really true.

It’s all just an implementation Detail

So a few people took issue with the fact that I was talking about the specifics of how objects and classes are actually implemented, rather than talking about them in an abstract sense. I think this has missed the point. I am not quibbling with how objects are implemented. My point is that objects don’t give any extra functionality beyond using dictionaries. I brought up the implementation as a way to demonstrate this. My point doesn’t rest on the fact that there is an actual dictionary internal to objects. It rests on the fact that objects are really just a collection of attributes with string names, these attributes can be added, deleted and modified at run time, and that there really isn’t anything more of value in the Python Object. The fact that this collection of attributes is implemented by a dictionary specifically isn’t important.

In particular some people stressed that what I was talking about was just an implementation detail unique to CPython, and that somehow it is different in some other Python implementation. I can’t stress this enough, it doesn’t matter what the actual implementation of the attributes of an object are, it is still bad. But also, no-one actually uses PyPy, Jython or IronPython. They are basically irrelevant.

Having methods defined on types is Convenient

I think this was probably the best point raised. I concede that if you want to maintain some state, and mutate it at runtime, it is convenient to wrap that state in an object and have specific methods defined on that object. The alternative is to just store your state using some built in types and operate on it with functions. However, what it seems you really want to do, restrict functions to only work with specific types, is to use static types. This seems like a roundabout way to achieve that. If you want static typing, you’re not going to be happy with Python. Really what you are doing here, is associating certain bits of data and functionality, like in an object oriented language, but without any guarantee of what the data actually is.

Python Objects are Different if you use __slots__

A lot of people brought up __slots__. Yes, if you use the __slots__ attribute Python objects work differently and my criticism doesn’t apply exactly. But, by default, python objects don’t use __slots__. The default implementation is actually pretty important, because it’s the one that is going to get used most often. In fact I’ve never seen someone use __slots__, I’ve only ever come across it as an obscure trick that gets recommended for python experts. It’s not a robust defence of a language feature to say, “actually there’s this special thing you can do that totally changes how it is implemented”. That’s really a sign that the default implementation is not good.

So __slots__ does change how objects work, and makes them a lot less like a special wrapper around a simple dictionary. But I don’t think it resolves any of the fundamental problems with using objects in Python. They are basically just a collection of attributes, we cannot say anything about them apart from how many attributes there are and what their names are. In particular they don’t specify the shape of the data.

One benefit of __slots__ is that your objects will use memory more efficiently. I would say, however, that you shouldn’t be worrying about memory optimisations like this in a high level language. Also, if there are memory optimisations in your language they shouldn’t be controlled in an opaque unintuitive way like the __slots__ attribute.