Tag Archives: hacks

metaclass service bus decorator concept

While playing around with some idea’s for making PyProxy completely overridable ( and also potentially undebuggable ) I started playing around with metaclasses.

Below is from my toxic directory [ gist url ]

from collections import defaultdict
from functools import wraps

Just some preliminary basic utilities

A simple bus implementation, decoratedCalls is a dictionary of service calls, with a list
of callbacks to each service call hook.

decoratedCalls = defaultdict(list)
 
def call(name, *args, **kwargs):
    print name, args, kwargs
    if name in decoratedCalls:
        for cb in decoratedCalls[name]:
            cb(*args, **kwargs)

Syntactic suger to add clarity later as to what functions are being bound to what. Adding in
debug/logging hooks here could allow for easier tracing of what is called for what methods.

class Subscribe(object):
    def __init__(self, name):
        self.name = name
 
    def __call__(self, f):
        decoratedCalls[self.name].append(f)
        return f

Here’s the actual bus decorator, very simple just wraps the decorated function with a pre and post bus calls.

 
class BusDecorator(object):
    def __init__(self, name):
        self.name = name
 
    def __call__(self, f):
 
        @wraps(f)
        def decorator(inst, *args, **kwargs):
            call("%s-pre" % self.name, inst, args, kwargs)            
            retval = f(inst, *args, **kwargs)
            call("%s-post" % self.name, inst, retval)            
            return retval
        return decorator

And here’s my Bus Metaclass that combines most of the above.

The ease of wrapping the target class is accomplished by cdict which is a dictionary of
every defined attribute of the target class. As you can see it’s trivial to spin
through and decorate every callable with the BusDecorator

 
class BusWrap(type):
 
    def __new__(mcs, clsname, bases, cdict):
 
        modName = cdict.get("__module__", "unknownclass")
 
        for name in cdict.keys():
            prefix = "%s.%s.%s" % ( modName, clsname, name)
            if callable(cdict[name]):                
                cdict[name] = BusDecorator(prefix)(cdict[name])
 
        return type.__new__(mcs, name, bases, cdict)

Now give a dirt simple class like

 
class Foo(object):
    __metaclass__ = BusWrap
 
    def __init__(self):
        print "init'd"
 
    def bar(self):
        print "bar"
 
    def blah(self):
        print "blah"
 
    def ich(self):
        print "ich"
 
    def ego(self):
        print "lego"
 
    def say(self, *args):
        print "Saying ", args

And two service handlers to pre Foo.bar being called and after Foo.ego is called

 
@Subscribe("__main__.Foo.bar-pre")        
def preBar(inst, *args, **kwargs):
    if not hasattr(inst, "ext_info"):
        inst.ext_info = "Here"
 
@Subscribe("__main__.Foo.ego-post")
def postEgo(inst, *args, **kwargs):
    if hasattr(inst, "ext_info"):
        print "Extended info is ", inst.ext_info

Our test shows….

x = Foo()
x.bar()
x.blah()
x.ich()
x.ego()
x.say("abba", "dabba")

this as output

__main__.Foo.__init__-pre (<__main__.__init__ object at 0x02637890>, (), {}) {}
init'd
__main__.Foo.__init__-post (<__main__.__init__ object at 0x02637890>, None) {}
__main__.Foo.bar-pre (<__main__.__init__ object at 0x02637890>, (), {}) {}
bar
__main__.Foo.bar-post (<__main__.__init__ object at 0x02637890>, None) {}
__main__.Foo.blah-pre (<__main__.__init__ object at 0x02637890>, (), {}) {}
blah
__main__.Foo.blah-post (<__main__.__init__ object at 0x02637890>, None) {}
__main__.Foo.ich-pre (<__main__.__init__ object at 0x02637890>, (), {}) {}
ich
__main__.Foo.ich-post (<__main__.__init__ object at 0x02637890>, None) {}
__main__.Foo.ego-pre (<__main__.__init__ object at 0x02637890>, (), {}) {}
lego
__main__.Foo.ego-post (<__main__.__init__ object at 0x02637890>, None) {}
Extended info is  Here
__main__.Foo.say-pre (<__main__.__init__ object at 0x02637890>, ('abba', 'dabba'), {}) {}
Saying  ('abba', 'dabba')
__main__.Foo.say-post (<__main__.__init__ object at 0x02637890>, None) {}


Plugin’s for python

As the #1 google result for “python plugin”, the linked to blog post is extremely valuable for jump starting research into implementing your own plugin system ( like me ) or finding one that is viable for implementation in your project ( possibly like me ).

These resources highlight one reason why there is not a standard Python plug-in framework: there are a variety of different capabilities that a user may want, and the complexity of the framework generally increases as these new capabilities are added

http://wehart.blogspot.com/2009/01/python-plugin-frameworks.html

A friendlier asynchronous twisted web, the ghetto monkey patch way

UPDATE to the UPDATE – A cleaned up and more coherent example of txweb is here
UPDATE – Github repo here

I like twisted, and I like Cherrypy, unfortunately just like my militant atheist friends and my more spiritual friends neither seems to get along with the other.

What to do? MONKEY PATCH + GHETTO HACKING to the rescue!

Note, this is just a mockup of CherryPy’s routing system and not a bridge or interface to CherryPy. There is no CherryPy to be had here, just ghetto py.

 
from twisted.web import server, resource
from twisted.internet import reactor
 
 
def expose(func):
    func.exposed = True
    return func
 
class PageOne(object):
 
    def foo(self, request):
        return "Hello From PageOne Foo!"        
    foo.exposed = True
 
    @expose
    def delayed(self, request):
        def delayedResponse():
            request.write("I was delayed :( ")
            request.finish()
 
        reactor.callLater(5, delayedResponse)
        return server.NOT_DONE_YET
 
 
class PageTwo(object):
 
    @expose
    def index(self, request):
        return "Hello From PageTwo index!"
 
 
class Root(object):
 
    @expose
    def index(self, request):
        return "Hello From Index!"
 
    @expose
    def __default__(self, request):
        return "I Caught %s " % request.path
 
    pageone = PageOne()
    pagetwo = PageTwo()
 
class OneTimeResource(resource.Resource):
    """
        Monkey patch to avoid rewriting more of twisted's lower web
        layer which does a fantastic job dealing with the minute details
        of receiving and sending HTTP traffic.
 
        func is a callable and exposed property in the Root OO tree
    """
    def __init__(self, func):
        self.func = func
 
    def render(self, request):
        #Here would be a fantastic place for a pre-filter
        return self.func(request)
        #ditto here for a post filter
 
 
class OverrideSite(server.Site):
    """
        A monkey patch that short circuits the normal
        resource resolution logic @ the getResourceFor point
 
    """
    def checkAction(self, controller, name):
        """
            On success, returns a bound method from the provided controller instance
            else it return None
        """
        action = None
        if hasattr(controller, name):
                action = getattr(controller, name)
                if not callable(action) or not hasattr(action, "exposed"):
                    action = None
 
        return action
 
 
    def routeRequest(self, request):
        action = None
        response = None
 
        root = parent = self.resource
        defaultAction = self.checkAction(root, "__default__")
 
        path = request.path.strip("/").split("/")
 
 
 
        for i in range(len(path)):
            element = path[i]
 
            parent = root
            root = getattr(root, element, None)
            request.prepath.append(element)
 
            if root is None:                
                break
 
            if self.checkAction(root, "__default__"):
                #Check for a catchall default action
                defaultAction = self.checkAction(root, "__default__")
 
 
            if element.startswith("_"):
                #500 simplistic security check
                action = lambda request: "500 URI segments cannot start with an underscore"
                break
 
            if callable(root) and hasattr(root, "exposed") and root.exposed == True:
                action = root
                request.postpath = path[i:] 
                break
 
 
 
        else:
            if action is None:
                if root is not None and self.checkAction(root, "index"):
                    action = self.checkAction(root, "index")
 
 
        #action = OneTimeResource(action) if action is not None else OneTimeResource(lambda request:"500 Routing error :(")
        if action is None:
            if defaultAction:
                action = defaultAction
            else:            
                action = lambda request:"404 :("
 
        return OneTimeResource(action)         
 
 
 
 
    def getResourceFor(self, request):
        return self.routeRequest(request)
 
"""
    Twisted thankfully doesn't do any type checking, so a
    dumb OO graph is A-Okay here.  It will be assigned to
    site.resource
"""
dumb = OverrideSite(Root())
 
reactor.listenTCP(80, dumb )
reactor.run()

Slapped this together in about 30 minutes… so there is a HIGH probability that it is almost entirely edge cased! Still it does work ( for me ) and it doesn’t hijack too much of twisted’s core, so it could be viable with a lot of unit-testing love, some additional sanity checking logics, and maybe some well thought out refactoring.

The magic JS console thing

Source here – https://github.com/devdave/DevDave-s-miscellanious-bucket/tree/master/jsConsole

Just wanted to see what would be involved in making a terminal like UI for the web. Honestly there’s a few REALLY annoying bits, but with a bit of hackery it was pretty straight forward. I’ve found splitting up the keyCode/which between key press and key down events was a lot simpler then making some sort of jury rigged code to character map when trying to compute shifted keys.

Also, I put in a brutally ugly left to right parser that does auto-complete / tab completion for good measure. It’s got a very limited vocabulary but I could easily snap it into an asyncronous Ajax handler to bridge it with a much more powerful server side module.

Like all of my short term projects, no warranty or guarantee that it works is provided or implied.

PyProxy – Aka the development Helper proxy

Coming out of nothing and into supah doopa Alpha is finally a working proof of concept of my python web proxy. I don’t really want to talk about the asinine alternatives I’ve tried until I finally said “fuck it, time to go completely twisted!” Low and behold the actual proxy part is 4 lines of code, which is then expanded to maybe 20-30 to allow for overloading some lower level classes.

Originally a public announcement for this project would have been in August at the earliest, give me time to clean things up and go from proof of concept to working concept but apparently a lot of other people have similar thoughts and I figured it’s better to collaborate then compete.

So some quick notes:
The ultimate goal for PyProxy ( or whatever it ends up being named ) is to sit between a developer and a development server. The first and immediate idea for this was to automagically parse out Python mechanize scripts to replicate the traffic. These mechanize scripts could then be collected into a suite, marking other scripts as requirements ( example login process ). That alone would make it pretty easy to create full system under test unit-tests. The next idea was to add in regex or pattern based hooks that could allow a developer to dial in to a specific domain, or even a specific set of webpages.

After that, the idea was to just continually tack on support plugins and scripts, maybe tell PyProxy the name of the target application’s database, and if it’s MySQL, switch on the general log. This could allow for combining both mechanize scripts AND a SQLObject or SQLAlchemy powered unit-test suite to assert that the correct data was changed.

The final future idea was to make a Firefox/Chrome extension that would allow a developer to control some parts of the proxy from their browser and also see additional information. For Python and PHP web apps, imagine have a finalization plugin that appended a response header listing all File’s used to perform a request…. then imagine having a “click to edit” button that, if the dev. instance is workstation local, would have your favorite IDE open the specified file for editing.

All in all, I think these are really subtle idea’s that if combined together, would cut down some mudane parts of developing a web app.

GitHub repo (https://github.com/devdave/PyProxy) here

A web-console implemented in nodeJS

Reading over this ( http://gonzalo123.wordpress.com/2011/05/16/web-console-with-node-js/ ) , I felt this was a pretty good and relatively real world example of nodeJS’s capabilities. I did find one line near the end very poignant and reflective of my own opinions at the end of the show & tell post

I don’t know if node.js is the future or is just another hype, but it’s easy. And cool. Really cool.

Canvas pixel collision detection

So… a fair number of people arrive here looking for how to detect when a pixel/object/thing has collided with something else.

I’ll add more information in the future but in the interim, the quick and dirty advice I can provide:

Given p = x, y where this is the point your trying to determine if it intersects another… the best place is to start with bounding box logic.  Regardless of what shape your object is, if wrap it in an invisible box boundry, it’s relatively trivial to do:

box upper left origin is bX, bY off size sizeX, sizeY.   If p.x greater then bX and p.x less then bX + sizeX AND p.y greater then bY and p.y less then bY + sizeY   you’re inside the bounding box and it’s highly likely your point has or will soon be colliding with an object.

That’s pretty much it and is perfect if you don’t mine lagging the crap out of whatever is running this logic.    To make these collision checks performant, you need to make the computer do less work.   For two dimensional space, my personnel research has led me to the Quad tree structure ( wikipedia article ).  I’ve got a brute force / naive implementation prototyped  @ here & here but as of 2011 March 9th it is not finished and needs some more work.

Basically Quad tree’s provides a super quick way for you to determine if there is any other objects in relative space to an entity.  Doing a quick lookup, if there’s nothing nearby then you can skip the bounding box checks and then the stupidly expensive perimeter intersection checks.

Full library is here and like most of my pet projects is heavily undocumented and completely absent of unit-testing.

Stellar simulator thingy

Inspired by this
IF YOU CAN SEE THIS, YOUR BROWSER SUCKS!





Source code @ Github here

So… this basically covers elliptical paths ( which I want to play with more ), a rendering chain I’m happy with, and a few others things. What isn’t exactly right: speed, planetary/stellar surface animation is missing, and I’d like to have a fly by mechanism that shifts the Y ratio progressively over time.

A DRY abomination

One of my habits when presented with a new framework/platform is to implement a problem I know like the back of my hand. Usually the user interface isn’t of concern so I haven’t done much to improve on the logic there. I decided to try something different with this implementation of Tic-Tac-Toe:

Given the following

        <table border="1" style="height:100%; width:100%" >
            <tr class="x0">
                <td class="y0">&nbsp;</td>
                <td class="y1">&nbsp;</td>
                <td class="y2">&nbsp;</td>
            </tr>
            <tr class="x1">
                <td class="y0">&nbsp;</td>
                <td class="y1">&nbsp;</td>
                <td class="y2">&nbsp;</td>
            </tr>
            <tr class="x2">
                <td class="y0">&nbsp;</td>
                <td class="y1">&nbsp;</td>
                <td class="y2">&nbsp;</td>
            </tr>
        </table>

I used the following logic to determine the coordinates from a user click

var clickHandler = function(e){
//Recieves a plan/jQuery managed event object                    
       var element = e.target;
       var parent  = element.parentNode;                    
       var x,y;
       //Below is fine for 0-9 scenario's but will need to be refactored for more
       //advanced/larger grids.
       try{
            y = element.classNames().detect(function(cls) { if( /y\d/.match(cls)){ return true; }})[1];
            x = parent.classNames().detect(function(cls) { if( /x\d/.match(cls)){ return true;}})[1];
        }catch(e){
            console.debug(e);
            return;
        }
 
         console.log("Coordinate " + x + "," + y);
};

The cool thing about this, is that with jQuery I can do jQuery(“#myTable tr.x2 td.y0”) and get the exact cell I am looking for… or even crazier stuff like:
“#myTable td.y0” to select vertically down a row
OR
“#myTable tr.x0” to select horizontally.