I’ve been wanting to roll out this post for quite some time but it kept evading me. In fact I had bulk of this post written out a long time back, just finishing it up was missing.
So, in this post I will be talking about how to deal with datetime
Python and how to interact with them when you are storing
Postgres. The aim of this post is to avoid all error
prone methods of timezone conversion of datetime objects as well as
those of storing them.
Datetime objects are probably one of the most used in any application. One obvious thing you’d need when designing a model, is probably the date of creation field. It actually gets trickier to tackle them as your models mutate1 after a while, and as surprising as it may sound, it is also fairly easy to miscalculate your datetimes, especially when you are handling multiple timezones. Ignoring this will bite you back hard someday.
So what’s so complicated about a simple datetime?
First a little bit of background before we dive in. Their are two
datetime objects. Timezone aware and Timezone
unaware or Naive objects. You can skip this part if you know the
difference between the two.
A timezone aware object is one which has a timezone information associated to it. That is, looking at such an instance, we will be able to find out the corresponding time in another time zone. Also, at any given moment in time, we can look this object up and identify which exact moment in time (seconds since epoch2) does it represent.
On the other hand, a timezone unaware or a naive object has no clue what moment in time it represents.
So if I am in India and say that the time is 10 AM, the person sitting next to me will know that it is 10 in the morning here in India. S/He can correctly identify it because they are also in India along with me. But if I say the same thing to a friend over the phone and ask them to second guess my location, s/he will surely be not able to answer my question. However, if I say that the current time is 10 AM, IST, my friend should be able to identify that I am somewhere in India, precisely from the timezone information I supplied. The analogy should be clear by now. I advise to read this section again if it isn’t.
What does Postgres want?
Postgres will store any timezone aware datetime in UTC but will display it in the time zone of the server, session or the user.
You can inspect the
timezone setting for postgres by logging
psql shell and running the following command:
dhanush=# show timezone; TimeZone -------------- Asia/Kolkata (1 row)
To set a custom timezone, use the following command:
dhanush=# set timezone='UTC'; SET dhanush=# show timezone; TimeZone ---------- UTC (1 row)
Here’s the same set of records in both
dhanush=# select created_at from users; created_at ---------------------------------- 2014-09-11 12:14:33.867216+05:30 2014-09-15 12:23:27.384904+05:30 2014-09-15 12:24:29.668802+05:30 2014-09-19 18:27:27.426808+05:30 2014-09-23 18:18:37.022816+05:30 2014-09-25 13:04:04.779181+05:30 2014-10-16 18:30:14.939262+05:30 (7 rows)
dhanush=# select created_at from users; created_at ------------------------------- 2014-09-11 06:44:33.867216+00 2014-09-15 06:53:27.384904+00 2014-09-15 06:54:29.668802+00 2014-09-19 12:57:27.426808+00 2014-09-23 12:48:37.022816+00 2014-09-25 07:34:04.779181+00 2014-10-16 13:00:14.939262+00 (7 rows)
So it is clear that postgres shows you any timezone aware datetime in the timezone that is set.
At this point, I would like to make it clear that I did NOT research about timezone unaware datetime entries in postgres.
Okay, so what does Python want?
Python supports both
naive objects as well as
objects. But, it is advised to never use timezone naive datetime
objects. The reason being, that a naive datetime object gives the end
user no information about the moment in time that it represents.
Datetime objects become more useful when they contain a timezone.
Hello pytz, bye bye errors!
pytz is an incredibly useful library
for handling timezones in
python. It does most of the work for you.
I have a naive datetime object. Please help me!
Take a look at the snippet below and the error that it produces:
In : import pytz In : from datetime import datetime In : now = datetime.now() In : now Out: datetime.datetime(2014, 10, 16, 14, 17, 51, 720507) In : ist = pytz.timezone('Asia/Kolkata') In : ist.localize(now) Out: datetime.datetime(2014, 10, 16, 14, 17, 51, 720507, tzinfo=<DstTzInfo 'Asia/Kolkata' IST+5:30:00 STD>) In : now.astimezone(ist) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-19-6bce1c9dbf37> in <module>() ----> 1 now.astimezone(ist) ValueError: astimezone() cannot be applied to a naive datetime
In the last command you can see that
astimezone works on
objects where as
localize works on
naive objects. The error
message is self explanatory so I wouldn’t go any deeper into this.
So, why don’t you use datetime.datetime.replace instead?
One common error in converting
naive objects to
aware ones is the
datetime.datetime.replace. Here’s a short example
demonstrating why it isn’t ideal.
In : ist Out: <DstTzInfo 'Asia/Kolkata' LMT+5:53:00 STD> In : stupid_utc_now = datetime.utcnow() In : stupid_utc_now Out: datetime.datetime(2014, 10, 16, 14, 26, 57, 150167) In : stupid_utc_now.replace(tzinfo=ist).utcoffset() Out: datetime.timedelta(0, 21180)
If you look carefully, the information about
STD which you might think is incorrect as the offset of
+5:30 and not
5:53. Bug? Not really. Initially
declared to be
+5:53 ahead of
UTC which was corrected later on to
replace doesn’t really know about this and thus you
can see the offset as
21180 seconds instead of
Now take a look at the following snippet:
In : intelligent_utc_now = datetime.utcnow() In : ist.localize(intelligent_utc_now).utcoffset() Out: datetime.timedelta(0, 19800) In : old_days = datetime(1700, 6, 18) In : old_days Out: datetime.datetime(1700, 6, 18, 0, 0) In : ist.localize(old_days) Out: datetime.datetime(1700, 6, 18, 0, 0, tzinfo=<DstTzInfo 'Asia/Kolkata' LMT+5:53:00 STD>)
Now carefully note the two offsets of
intelligent_utc_now. You can
see the correct offset this time. Whereas, for
localized to IST, the offset is
5:53. That’s the wonder of
Depending on the time, it returns the offset accordingly.
a really old date and
+5:30 for a date after the time that the
offset for IST was corrected.
Finally, if you want to work with the current time as a timezone aware object, which once again I highly advise you to use instead of a naive timezone object, you should do the following:
In : datetime.now(pytz.timezone('Asia/Kolkata')) Out: datetime.datetime(2014, 10, 20, 13, 25, 14, 121485, tzinfo=<DstTzInfo 'Asia/Kolkata' IST+5:30:00 STD>)
And, if you need the current time in UTC,
In : datetime.now(pytz.utc) Out: datetime.datetime(2014, 10, 20, 7, 55, 17, 817783, tzinfo=<UTC>)
In : pytz.utc.localize(datetime.utcnow()) Out: datetime.datetime(2014, 10, 20, 7, 55, 23, 25311, tzinfo=<UTC>)
But in my opinion the first approach looks way more clean. For fans of
datetime.datetime.replace, you can still use it for creating a
datetime object in UTC without any problems.
- Always use
- When dealing with
localizeto add timezone
- When you are converting timezones use
datetime.datetime.replacefor adding or manipulating timezone information on a
datetimeobject. Except when you use it to instantiate a datetime object in UTC.
 Read this post here if you want to know why I say your models mutate instead of change.comments powered by Disqus