We had a situation come up at work the other day where we seemed to be receiving some spurious data from a data provider. The data providing process connects to one of our processes on some specific port and sends a relentless stream of data. Said spurious data uncovered a bug in our process which was causing it to die abnormally. My first action was to get tcpdump to show me what was going on. But the results were just wrong. I suspect the incorrect results I was seeing were caused by the antiquated version of tcpdump running on an antiquated FreeBSD machine and trying to view the results on Wireshark/Ethereal on a recent Ubuntu box.
So, I figured a logging proxy help. So I whipped one up using Twisted. It worked pretty well. I know I didn’t get everything right since I don’t regularly use Twisted. Basically, every connection that is made to the proxy from the data provider initiates a client connection to my server process. That connection also creates a log file on disk with a name that identifies where the connection came from. Each piece of data that is sent from the data provider is logged and forwarded on to my process. What it doesn’t handle properly is my process going down. I didn’t spend enough time to figure out exactly how to attach a reference to the server piece of the proxy onto the client piece.
Enter the recipe of the week. Just ten days ago, this excellent recipe was either submitted or updated (I can’t tell which). This recipe contains code for a proxy server which would fit my needs and log a hexdump of the received data. I haven’t tried it in the context of what I was trying to do, but given the testing I did with it, it looks like it would work quite nicely. The only thing I would change is the format of the logging. For my purposes, I’d still need to have a raw log of the transmitted data. But this is a great recipe that shows an example of a working proxy in Twisted.