Finding Time Intervals Per Day From A List Of Timestamps In Python

i am trying to compute time intervals per day from a list of unix timestamps in Python. I have searched for simular questions on stack overflow but mostly found examples of computi

Solution 1:

You can use collections.defaultdict. It's amazingly handy when you're trying to build a collection without inital estimates on size and members.

from collections import defaultdict

# Initialize default dict by the type list# Accessing a member that doesn't exist introduces that entry with the deafult value for that type# Here, when accessing a non-existant member adds an empty list to the collection
intervalsByDate = defaultdict(list)

for t in timestamps:
    dt = dt.datetime.fromtimestamp(t)
    myDateKey = (dt.day, dt.month, dt.year)
    # If the key doesn't exist, a new empty list is added
    intervalsByDate[myDateKey].append(t)

From this, intervalsByDate is now a dict with values as a list timestamps sorted based on the calendar dates. For each date you can sort the timestamps and get the total intervals. Iterating the defaultdict is identical to a dict (being a sub-class of dicts).

output = {}
fordate, timestamps in intervalsByDate.iteritems():
    sortedIntervals = sorted(timestamps)
    output[date] = sortedIntervals[-1] - sortedIntervals[0]

Now output is a map of dates with intervals in miliseconds as the value. Do with it as you will!

---

EDIT

Is it normal that the keys are not ordered? I have keys from different months mixed togheter.

Yes, because (hash)maps & dictsare essentially unordered

How would I be able to, for example, select the first 24 days from a month and then the last

If I was very adamant on my answer, I'd maybe look at this, which is an Ordered default dict.. However, you could modify the datatype of output to something which isn't a dict to fit your needs. For example a list and order it based on dates.

Solution 2:

If the list sorted as in your case then you could use itertools.groupby() to group the timestamps into days:

#!/usr/bin/env pythonfrom datetime import date, timedelta
from itertools import groupby

epoch = date(1970, 1, 1)

result = {}
assert timestamps == sorted(timestamps)
for day, group in groupby(timestamps, key=lambda ts: ts // 86400):
    # store the interval + day/month in a dictionary.
    same_day = list(group)
    assertmax(same_day) == same_day[-1] andmin(same_day) == same_day[0]
    result[epoch + timedelta(day)] = same_day[0], same_day[-1] 
print(result)

Output

{datetime.date(2007, 4, 10): (1176239419.0, 1176239419.0),
 datetime.date(2007, 4, 11): (1176334733.0, 1176334733.0),
 datetime.date(2007, 4, 13): (1176445137.0, 1176445137.0),
 datetime.date(2007, 4, 26): (1177619954.0, 1177621082.0),
 datetime.date(2007, 4, 29): (1177838576.0, 1177838576.0),
 datetime.date(2007, 5, 5): (1178349385.0, 1178401697.0),
 datetime.date(2007, 5, 6): (1178437886.0, 1178437886.0),
 datetime.date(2007, 5, 11): (1178926650.0, 1178926650.0),
 datetime.date(2007, 5, 12): (1178982127.0, 1178982127.0),
 datetime.date(2007, 5, 14): (1179130340.0, 1179130340.0),
 datetime.date(2007, 5, 15): (1179263733.0, 1179264930.0),
 datetime.date(2007, 5, 19): (1179574273.0, 1179574273.0),
 datetime.date(2007, 5, 20): (1179671730.0, 1179671730.0),
 datetime.date(2007, 5, 30): (1180549056.0, 1180549056.0),
 datetime.date(2007, 6, 2): (1180763342.0, 1180763342.0),
 datetime.date(2007, 6, 9): (1181386289.0, 1181386289.0),
 datetime.date(2007, 6, 16): (1181990860.0, 1181990860.0),
 datetime.date(2007, 6, 27): (1182979573.0, 1182979573.0),
 datetime.date(2007, 7, 1): (1183326862.0, 1183326862.0)}

If there is only one timestamp in that day than it is repeated twice.

how would you afterwards do to test if the last (for example) 5 entries in the result have a larger interval than the previous 14?

entries = sorted(result.items())
intervals = [(end-start) for _, (start, end) in entries]
print(max(intervals[-5:]) >max(intervals[-5-14:-5]))
# ->False

Solution 3:

Just subtract the 2 dates from each other. This will result in a timedelta instance. See datetime.timedelta: https://docs.python.org/2/library/datetime.html#timedelta-objects

from datetime import datetime
delta = datetime.today() - datetime(year=2015, month=01, day=01)
#Actual printed out values may change depending o when you execute this :-)print delta.days, delta.seconds, delta.microseconds #prints 49 50817 381000 print delta.total_seconds() #prints 4284417.381 which is 49*24*3600 + 50817 + 381000/1000000

Combine this with row slicing and zip to get your solution. An example solution would be:

timestamps = [1176239419.0, 1176334733.0, 1176445137.0, 1177619954.0, 1177620812.0, 1177621082.0, 1177838576.0, 1178349385.0, 1178401697.0, 1178437886.0, 1178926650.0, 1178982127.0, 1179130340.0, 1179263733.0, 1179264930.0, 1179574273.0, 1179671730.0, 1180549056.0, 1180763342.0, 1181386289.0, 1181990860.0, 1182979573.0, 1183326862.0]
timestamps_as_dates = [datetime.fromtimestamp(int(i)) for i in timestamps]
# Make couples ofeachtimestampwith the next one
# timestamps_as_dates[:-1] ->all your timestamps but the lastone
# timestamps_as_dates[1:]  ->all your timestamps but the firstone
# zip them together so that firstandsecondareone couple, thensecondand third, ...
intervals = zip(timestamps_as_dates[:-1],timestamps_as_dates[1:])
interval_timedeltas = [(interval[1]-interval[0]).total_seconds() forintervalin intervals]
# result= [95314.0, 110404.0, 1174817.0, 858.0, 270.0, 217494.0, 510809.0, 52312.0, 36189.0, 488764.0, 55477.0, 148213.0, 133393.0, 1197.0, 309343.0, 97457.0, 877326.0, 214286.0, 622947.0, 604571.0, 988713.0, 347289.0]

This also works for adding or subtracting a certain period from a date:

from datetime import datetime, timedeltatomorrow= datetime.today() + timedelta(days=1)

I don't have an easy solution for adding or subtracting months or years.