Get Database Cursor From Django's Rawqueryset

I have a sizable table (20M+) in Postgres am I try to do a raw Django query on it: tweets = TweetX.objects.raw('SELECT * from twitter_tweet').using('twittertest') I get a RawQuery

Solution 1:

It seems that it is rather difficult to persuade django/postgres to use database cursors. Instead it fetches everything and then put a client side iterator (called cursor) over it.

Found a solution overhere that explicitly creates a db cursor. Only downside is it does not fit into django models anymore.

from django.db import connections

conn = connections['twittertest']
# This is required to populate the connection object properlyif conn.connection isNone:
    cursor = conn.cursor()        

cursor = conn.connection.cursor(name='gigantic_cursor')
cursor.execute("SELECT * from twitter_tweet")

for tweet in cursor:
    #profit