Django / Wsgi - How To Profile Partial Request? My Profiling Tools Are Per-request But App Runs Out Of Memory Before Then

How can I profile my python / django application which is crashing on a single request after 100 seconds of hogging more memory? All I see in top is that the wsgi process is consum

Solution 1:

A bit fiddly and will have some overhead, but you could use sys.setprofile() to provide a function to be called on entry and exit from functions and dump progress of calls out to a log file yourself, potentially with check of memory in use at the same time.

http://docs.python.org/dev/library/sys.html#sys.setprofile

Also perhaps check out heapy as a way of getting console type access into your live process to dump out memory/object usage.