Women in Technology

Hear us Roar



Article:
  Untwisting Python Network Programming
Subject:   performance
Date:   2006-08-14 08:16:53
From:   radix
Response to: Why untwisting?

How were these numbers reached? Can you show us any python code or commands that you used to find them?


I notice that your synchronous version of the telnet code immediately closes the socket whereas the twisted version is waiting for the *server* to end the connection; this could definitely be affecting your times. If you add a transport.loseConnection call after your write call, the semantics should line up better, and I imagine performance will be closer to what we expect.


Also, why are you calling the private "_write" method in the telnet example?

Full Threads Oldest First

Showing messages 1 through 1 of 1.

  • performance
    2006-08-17 04:57:06  Kendrew [View]

    The time includes starting the interpreter and doing the whole script. It is true that the import statements (and any networking delay) contributes the majority to the execution time in these small programs. In fact, before measuring I did some tidying of the import statements so that only required items are imported before use.

    For the telnet, I measure again with the addition of transport.loseConnection() and using StatefulTelnetProtocol instead of Telnet. The Twisted telnet runs faster than before, as expected:


    Using core modules:
    start server (no net): 0.5809 sec
    send mails (smtp): 1.3601 sec
    view mails (pop3): 0.7007 sec
    delete mails (pop3): 0.5187 sec
    stop server (telnet): 0.5124 sec

    Using Twisted:
    start server (no net): 0.5959 sec
    send mails (smtp): 2.2488 sec
    view mails (pop3): 1.4274 sec
    delete mails (pop3): 1.3074 sec
    stop server (telnet): 1.3213 sec


    If you're interested, there is the Python program to measure the timing. It just invokes various usages of the two networking programs and takes the average.


    #!/usr/bin/python
    # file: mail-timeit.py
    # Measures the timing of invoking mail-core.py and mail-twisted.py

    from time import sleep
    from timeit import Timer
    cmdline = ''

    def doit(cmd, arg, array, rest):
    global cmdline
    cmdline = cmd + ' ' + arg
    print; print cmdline
    array.append(Timer('os.system(cmdline)',
    'import os; from __main__ import cmdline').timeit(1))
    sleep(rest)

    def dostat(cmd, times):
    stat = [[], [], [], [], []] # for 1, s, v, d, 0

    for i in range(times):
    doit(cmd, '1', stat[0], 12)
    doit(cmd, 's', stat[1], 5)
    doit(cmd, 'v', stat[2], 1)
    doit(cmd, 'd', stat[3], 1)
    doit(cmd, '0', stat[4], 2)

    return stat

    def avgstat(stat, fr, to):
    return [ sum(i[fr:to]) / (to-fr) for i in stat ]

    def printavgs(avgs):
    labels = [
    'start server (no net)',
    'send mails (smtp)',
    'view mails (pop3)',
    'delete mails (pop3)',
    'stop server (telnet)']
    for i, j in zip(labels, avgs):
    print '%25s: %.4f sec' % (i, j)

    if __name__ == '__main__':
    times = 11

    stat1 = dostat('mail-core.py', times)
    avgs1 = avgstat(stat1, 1, times)

    stat2 = dostat('mail-twisted.py', times)
    avgs2 = avgstat(stat2, 1, times)

    # print stat1
    printavgs(avgs1)
    # print stat2
    printavgs(avgs2)

    # end of mail-timeit.py