Refactored scope

http://www.stevecoursen.com/209/stackless-python-meets-twisted-matrix/

https://github.com/smira/txZMQ

https://pypi.python.org/pypi/Twistless/1.0.0

goals: evaluate speed over memory for pypy using stackless/coroutines with ZMQ and twistless ( if its even possible without rebuilding txZMQ ).

Initial goal would be to take github.com/devdave/pyfella and let two stacks fight it out for an hour ( assuming 10x deep othello minimax ).

CQL3 is a very nice abstraction to Cassandra but its important to pay attention to what it is doing.

In SQL land, 1 record == 1 row. In Cassandra 1 record == 1 row, but 2+ records can ALSO be on the same row. This has to do with CQL’s partition and primary keys. Your partition key is what decides which row a record belongs to while the primary key is where the record is in a row. If you only have a primary key and no partition key, 1 record == 1 row, but if you have a composite ( partition key, primary key) every record where partition key is the same is going on the same row.

I had a few rows that were ~30GB in size which put stress on nodes using m1.xlarge ( 8GB heap, 300MB new heap size ) with epic Compaction cycles of doom.

These directions will setup openvpn ( you need that unless your dev’ing inside AWS )

http://sysadminandnetworking.blogspot.com/2012/12/openvpn-on-ec2aws.html

New Python cassandra driver is here https://github.com/datastax/python-driver
It’s really new so it has sharp edges.

Initially I was going to set the experiment with ZMQInline aside but I am starting to realize that it does have some merits.

Below is a burnout test I did using dotGraph ( current name candidate )

@ZMQInline()
    def do_kickstart(self):
        self.log.debug("do_kickstart")
        for i in range(1, 100):
            try:
                try:
                    self.log.debug("Process Loop {0}".format(i))
                    response = yield Request(self.control, RequestKickstart(), timeout = 5000)
                except TimedoutError:
                    self.log.exception("Timedout")
 
                except Exception:
                    self.log.exception("Unexpected exception")
                else:
                    self.log.debug("Got "+ repr(response))
 
                    if RespondWithKickstart.Matches(response):
                        self.log.debug("Got Kickstart!")
            except Exception as e:
                self.log.error("Caught an exception leak {0}".format(e))
 
        self.loop.stop()

In Dotgraph, Kickstart is the first task both client’s and services need to accomplish and it’s basically “Hello I am here, who do I talk to?”. Depending on the Kickstart response, clients and services might be sent to a completely different host, port set. The idea is that if DotGraph needs to grow, it can have an array of slave hubs and then delegate new client’s to the most underbooked.

One thing I realized is that the above structure would perfect for doing standoff logic ( attempt, time out…wait, attempt N seconds later). Of course it begs the question of WHY the other party hasn’t responded in time ( are they down or just backlogged? ). Other problem is how to do unit-testing cleanly? Still, I haven’t thrown it away yet so maybe I will find a use for it in DotGraph.

From stack exchange – http://programmers.stackexchange.com/questions/185923/how-can-i-deal-with-a-team-member-who-dislikes-making-comments-in-code/185926#185926

I think anyone who has written software alongside anyone else has had at least one grievance with their peers work. Sometimes it can be benign like

if(foo == bar): do_stuff

versus

if(foo == bar) {
   do_stuff
}

And other times it can a tad more annoying like

class Foo { function Foo(arg1,arg2){ this.arg1=arg1, this.arg2=arg2} }

To get a co-worker to change that style, from what I’ve seen it’s not going to happen by asking them to use more XXX and formatting politely. I worked with someone like this and they got immediately defensive and said “I just don’t like wasting space.” which kind of took me off guard as it didn’t really make any sense. For that case I was able to pull rank and kickback their commit’s which probably isn’t the best solution. I’ve seen other solutions like implementing a CI that stage one was an automatic style checker… which is basically what I did but automated and probably a worse solution as sometimes you need to write something really dirty. Maybe a better solution is code reviews but it is fairly rare to see a company institute those. 9 out of 11[SIC] companies, the guy writing the checks for the software engineers isn’t an engineer themselves and in the short term they’re going to see that one of their senior engineers isn’t directly doing work that correlates to profits. In that last case, when it comes up in a review the lead can justify the expenses by pointing out that they’re minimizing the wasted time other staff have to spend just to comprehend something.

Finally at the end of the day, while it’s important that a person finds reward in their job… it’s still a job and not your private playpen. Instead it’s more like software survivor island, the majority has to work together to keep their company/product going and if one goof ball insists on writing software in a conflicting manner then the majority it doesn’t really matter if they are “right”, they need to conform or GTFO.

For scenario’s where you ARE that goofball, you need to get political and I don’t mean selling the highest non-technical person on your island but demonstrating to your team the merits of an idea and working with their feedback. For real world example, I joined a consultancy as the goofball and started with the company architect and development manager. I didn’t sell them on the idea that I was right but on the idea until it became their “right” idea. From there I prodded momentum until one day the idea stopped being a minority opinion and a coup was staged. A week later the company made a fairly dramatic shift in how they did development. That’s a nice happy ending scenario but it doesn’t always happen. Twice I’ve failed to sell the virtues of unit-testing ( one case they assigned me the task of writing unit-tests which was just a means of getting me to shut up about them… but completely negated the idea) and twice I walked away because I got tired of having last minute “Oh my god everything is fucked up” hack sessions to fix prod. That’s life, sometimes you lose and sometimes you win.