Downloading The File And Cancel In Python

I'm working on my python script to download a xml file from my server and write the data in a sqlite3 database in parallel. I need some help regarding how to cancel the connection

Solution 1:

To download a file in the background, while processing events in the foreground, it's much simpler to use a Process. They have better Python support, they can do CPU in addition to I/O, and are killable if they act up.

The following starts a background process. The parent's main loop does its own thing, briefly checking if the download is done every now and then. If download is done, it continues processing, then exits.

Have fun!

source

import logging, multiprocessing, time, urllib2

def download(url):
    mylog = multiprocessing.get_logger()
    mylog.info('downloading %s', url)
    time.sleep(2)
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    data = response.read()
    response.close()
    # more here
    time.sleep(3)
    mylog.info('done')

def main():

    mylog = multiprocessing.log_to_stderr(level=logging.INFO)
    mylog.info('start')

    download_proc = multiprocessing.Process(
        target=download,
        args=('http://example.com',),
        )
    download_proc.start()

    while True:
        mylog.info('ding')
        if download_proc.join(timeout=0.1) is None:
            mylog.info('download done!')
            break
        time.sleep(1)

    mylog.info('done!')

if __name__=='__main__':
    main()

output

[INFO/MainProcess]start[INFO/MainProcess]ding[INFO/Process-1]childprocesscallingself.run()
[INFO/Process-1]downloadinghttp://example.com[INFO/MainProcess]downloaddone!
[INFO/MainProcess]done!
[INFO/MainProcess]processshuttingdown[INFO/MainProcess]callingjoin() forprocessProcess-1[INFO/Process-1]done[INFO/Process-1]processshuttingdown[INFO/Process-1]processexitingwithexitcode0