The poster asked “This may be offtopic but i really want to share this.
Did you ever faced criticism with your code. How to overcome this criticism ?”

And this was my response.

Every so often I go back to something I wrote 10 years ago and I laugh my ass off. Now if 10 years ago someone had done that, it would have been somewhat of a painful experience. People have already mentioned “you are not you’re code” which is mostly true BUT at same time you are your code at this moment in time. When a peer criticizes your code, you need to quickly detach yourself and hear out what they have to say. Sometimes their input is going to be asinine ( “I prefer naming variable after my kid’s” ) but hopefully its going to real value ( “You should implement dash or camel casing, does ‘expertsexchange’ mean Expert Sex change or Experts exchange? “)

Before I give whatever advice on filtering poisonous vs constructive criticism, going to address why you want criticism. If you are a small person, you will be stuck in a small world and only realize the situation when people are selling cars and you’re still crafting buggy whips to a doomed industry ( eg COBOL & mainframes vs Java & server farms ). To improve your craft, yes you need challenging work but you also need to be exposed to different idea’s or you will not grow professionally.

Now as far as handling criticism and deciding if it has value. #1 is that you need to check your emotions and listen. #2 identify the problem they have with your work and clarify so you have concrete examples of what is and is not the problem. #3 Evaluate the value of fixing the problem “Does this improve things for my team and our success” or “Does this make my product better and or more maintainable?”

If the person cannot give concrete examples of #2, tell them you don’t understand. If they cannot do this without resorting to verbal abuse, conversation is over and escalate to supervision or discontinue discourse.

If the person cannot demonstrate #3 ( eg how does using their children’s names make things “better”? ) escalate or discontinue, telling them that you don’t seen an advantage to their proposal.

Finally, emotion’s get you into a fight you may not be able to win or will have costs to your career down the line. One example is a person I found immensely influential to my career and I thought he was the bee’s knees. This person was outspoken and somewhat vitriolic but he was generally right ( or appeared to be ). 5 years down the road, he seems like a “has been” that is constantly verballing threatening to beat up people when they criticize his code or his behavior “I am a MMA fighter, I will kick your ass”. For the most part I think that guy’s career is on its way out as who wants to collaborate or associate with him? Also I am not talking about Linus Torvald… he’s a completely different kind of crazy with a completely different problem.

http://www.stevecoursen.com/209/stackless-python-meets-twisted-matrix/

https://github.com/smira/txZMQ

https://pypi.python.org/pypi/Twistless/1.0.0

goals: evaluate speed over memory for pypy using stackless/coroutines with ZMQ and twistless ( if its even possible without rebuilding txZMQ ).

Initial goal would be to take github.com/devdave/pyfella and let two stacks fight it out for an hour ( assuming 10x deep othello minimax ).

I have one particular “job” that has 3 sub processes moving as fast as humanly possible to build a report. The main slowdown is an external data source which isn’t outright terrible but its not great either. The worst possible outcome is when this thing hangs or misses available work which it was predisposed to do a lot.

Various kill signals usually failed to give me an idea of where the workers were getting hung up and I wasn’t really excited about putting tracer log messages everywhere. Fortunately I have a dbgp enabled IDE and I found this answer on SO. http://stackoverflow.com/a/133384/9908

Taking that I modified it to look like this:

import traceback, signal
 
#classdef FeedUserlistWorker which is managed by a custom multiprocessing.Pool implementation.
 
    @classmethod
    def Create(cls, feed, year_month = None):
 
        signal.signal(signal.SIGUSR1, FeedUserlistWorker._PANIC)
 
        try:
            return cls(year_month=year_month, feed=feed).run()
        except Exception as e:
            from traceback import print_exc
            print_exc(e)
            sys.stderr.flush()
            sys.stdout.flush()

the print_exc is there because there isn’t a very reliable bridge to carry Exceptions from child to parent. Flush’s are there because stdout/stderr are buffered in between the parent pool manager.

    @classmethod
    def _PANIC(cls, sig, frame):
        d={'_frame':frame}
        d.update(frame.f_globals)
        d.update(frame.f_locals)
 
        from dbgp.client import brk; brk("192.168.1.2", 9090)

The only thing that matters is that call to dbgp. Using that tool, I was able to step up the call stack, fire adhoc commands to inspect variables in the stack frame, and find the exact blocking call, which turned out to be the validation/authentication part of boto s3. That turned out to be a weird problem as I had assumed the busy loop/block was in my own code ( eg while True: never break ), fortunately it has an easy fix https://groups.google.com/forum/#!msg/boto-users/0osmP0cUl5Y/5NZBfokIyoUJ which resolved the problem as my Pool manager doesn’t mark tasks complete and failures will only cause the lost task to be resumed from the last point of success.

CQL3 is a very nice abstraction to Cassandra but its important to pay attention to what it is doing.

In SQL land, 1 record == 1 row. In Cassandra 1 record == 1 row, but 2+ records can ALSO be on the same row. This has to do with CQL’s partition and primary keys. Your partition key is what decides which row a record belongs to while the primary key is where the record is in a row. If you only have a primary key and no partition key, 1 record == 1 row, but if you have a composite ( partition key, primary key) every record where partition key is the same is going on the same row.

I had a few rows that were ~30GB in size which put stress on nodes using m1.xlarge ( 8GB heap, 300MB new heap size ) with epic Compaction cycles of doom.

curl -L http://debian.datastax.com/debian/repo_key | sudo apt-key add -
sudo sh -c 'echo "deb http://debian.datastax.com/community/ stable main" >> /etc/apt/sources.list.d/datastax.list'
 
sudo apt-get update
sudo apt-get install cassandra

Really wish there was a cassandra-core and cassandra-server divide as more often than not I just need the c* tools and libraries and not so much the server itself.

Also 1.2.5 is TOXIC as all hell for high read/write environments.

These directions will setup openvpn ( you need that unless your dev’ing inside AWS )

http://sysadminandnetworking.blogspot.com/2012/12/openvpn-on-ec2aws.html

New Python cassandra driver is here https://github.com/datastax/python-driver
It’s really new so it has sharp edges.

Given a project structure like:

my_project
 main.py
 www/
    index.html
    scripts/ 
        jquery-1.9.1.min.js

main.py looks like:

import sys
from os.path import dirname, join
 
#from PySide.QtCore import QApplication
from PySide.QtCore import QObject, Slot, Signal
from PySide.QtGui import QApplication
from PySide.QtWebKit import QWebView, QWebSettings
from PySide.QtNetwork import QNetworkRequest
 
 
 
web     = None
myPage  = None
myFrame = None
 
class Hub(QObject):
 
    def __init__(self):
        super(Hub, self).__init__()
 
 
    @Slot(str)
    def connect(self, config):
        print config
        self.on_client_event.emit("Howdy!")
 
    @Slot(str)
    def disconnect(self, config):
        print config
 
    on_client_event = Signal(str)
    on_actor_event = Signal(str)
    on_connect = Signal(str)
    on_disconnect = Signal(str)
 
 
myHub = Hub()
 
class HTMLApplication(object):
 
    def show(self):
        #It is IMPERATIVE that all forward slashes are scrubbed out, otherwise QTWebKit seems to be
        # easily confused
        kickOffHTML = join(dirname(__file__).replace('\\', '/'), "www/index.html").replace('\\', '/')
 
        #This is basically a browser instance
        self.web = QWebView()
 
        #Unlikely to matter but prefer to be waiting for callback then try to catch
        # it in time.
        self.web.loadFinished.connect(self.onLoad)
        self.web.load(kickOffHTML)
 
        self.web.show()
 
    def onLoad(self):
        if getattr(self, "myHub", False) == False:
            self.myHub = Hub()
 
        #This is the body of a web browser tab
        self.myPage = self.web.page()
        self.myPage.settings().setAttribute(QWebSettings.DeveloperExtrasEnabled, True)
        #This is the actual context/frame a webpage is running in.  
        # Other frames could include iframes or such.
        self.myFrame = self.myPage.mainFrame()
        # ATTENTION here's the magic that sets a bridge between Python to HTML
        self.myFrame.addToJavaScriptWindowObject("my_hub", myHub)
 
        #Tell the HTML side, we are open for business
        self.myFrame.evaluateJavaScript("ApplicationIsReady()")
 
 
#Kickoff the QT environment
app = QApplication(sys.argv)
 
myWebApp = HTMLApplication()
myWebApp.show()
 
sys.exit(app.exec_())

and index.html looks like:

<html>
    <head>
        <!-- Oh hell no, I can access anything on my computer via this -->
        <script src="./scripts/jquery-1.9.1.min.js"></script>
 
        <script language="javascript" type="text/javascript">
 
            function callback(){
                console.log("callback!");
                alert("called!");
            }
 
 
            function sendStuff() {
                console.log("sendStuff!");
                my_hub.connect("hello?");
            }
 
            function ApplicationIsReady(){
                console.log("ApplicationIsReady!");
                my_hub.on_client_event.connect(callback);
            }
 
        </script>
    </head>
    <body>
        Tell the hub to connect
        <button onclick=sendStuff()>No, click me!</button>
    </body>
    <script>
        alert(typeof jQuery);
    </script>
</html>

Basically class Hub is a bridge/interface pattern between Python and the HTML Javascript engines. It’s probably best to pass things between the two as JSON as that’s a language both sides are readily prepared to deal with.

Stuff I didn’t deal with yet: The Main window title ( self.web.setWindowTitle), setting a window Icon ( no clue on this one ).

Otherwise this gives me an OS agnostic UI, I can leverage my HTML skillset without having to learn QML or QT Designer’s UI interface, and I can hopefully recycle some logic. Additionally I can go full fledged with the bridge logic, split it between the two, or shove all the complexity into JS and have basic Python endpoints exposed.

Initially I was going to set the experiment with ZMQInline aside but I am starting to realize that it does have some merits.

Below is a burnout test I did using dotGraph ( current name candidate )

@ZMQInline()
    def do_kickstart(self):
        self.log.debug("do_kickstart")
        for i in range(1, 100):
            try:
                try:
                    self.log.debug("Process Loop {0}".format(i))
                    response = yield Request(self.control, RequestKickstart(), timeout = 5000)
                except TimedoutError:
                    self.log.exception("Timedout")
 
                except Exception:
                    self.log.exception("Unexpected exception")
                else:
                    self.log.debug("Got "+ repr(response))
 
                    if RespondWithKickstart.Matches(response):
                        self.log.debug("Got Kickstart!")
            except Exception as e:
                self.log.error("Caught an exception leak {0}".format(e))
 
        self.loop.stop()

In Dotgraph, Kickstart is the first task both client’s and services need to accomplish and it’s basically “Hello I am here, who do I talk to?”. Depending on the Kickstart response, clients and services might be sent to a completely different host, port set. The idea is that if DotGraph needs to grow, it can have an array of slave hubs and then delegate new client’s to the most underbooked.

One thing I realized is that the above structure would perfect for doing standoff logic ( attempt, time out…wait, attempt N seconds later). Of course it begs the question of WHY the other party hasn’t responded in time ( are they down or just backlogged? ). Other problem is how to do unit-testing cleanly? Still, I haven’t thrown it away yet so maybe I will find a use for it in DotGraph.

A funny little quirk has gotten me twice now ( hopefully it won’t be three times ).

In Twisted, once the generator has been called ( even inadvertently ) it becomes the defacto instance.

In PyZMQ, calling ioloop.IOLoop() creates a new loop ( and blows the old one away )… The correct approach is to always call it via

    ioloop.IOLoop.Instance()

I don’t want to talk about how much time I wasted working out how to make a pyzmq like interface similar to Twisted’s inlineCallback, but it was enough!